Calculate similarity score between scene and template object
computer-vision, image-processing, ios, objective-c, opencv
Solution
A similarity score is greater if the object and scene are more similar (as opposed to a dissimilarity score, where a higher score means they are more dissimilar). Since you are using distances with FLANN (which I assume is giving you approximate euclidean distances between descriptors) a dissimilarity score is easier to generate, since euclidean distance is greater if descriptors are further apart in the descriptor space, and small if they are close together.
One simple way to generate a dissimilarity score would be to: 1. For each descriptor in the object image: calculate the minimum distance to each descriptor in the scene image. 2. Sum the (minimum) distances, and normalize by the number of descriptors in the object image.
Then you will have a single score quantifying the match between the object and the scene.
Problem
How can I calculate some comparable similarity score which tells me how similar the `img_scene` is compared to `img_object`. When I render the `img_matches`, the homography successfully renders the boundaries of the found object in the scene, but I need some comparable `score` like `if (score > THRESHOLD) { /* have match */ } else { /* dont have match */ }`. ``` Mat img_scene = srcImage; Mat img_object = _templateImage; //-- Step 1: Detect the keypoints using SURF Detector SurfFeatureDetector detector(_minHessian); std::vector<KeyPoint> keypoints_object, keypoints_scene; detector.detect(img_object, keypoints_object); detector.detect(img_scene, keypoints_scene); //-- Step 2: Calculate descriptors (feature vectors) SurfDescriptorExtractor extractor; Mat descriptors_object, descriptors_scene; extractor.compute(img_object, keypoints_object, descriptors_object); extractor.compute(img_scene, keypoints_scene, descriptors_scene); if (descriptors_object.type() != descriptors_scene.type()) return; //-- Step 3: Matching descriptor vectors using FLANN matcher FlannBasedMatcher matcher; std::vector<DMatch> matches; matcher.match(descriptors_object, descriptors_scene, matches); double max_dist = 0; double min_dist = 100; //-- Quick calculation of max and min distances between keypoints for (size_t i = 0; i < (size_t)descriptors_object.rows; i++ ) { double dist = matches[i].distance; if (dist < min_dist) min_dist = dist; if (dist > max_dist) max_dist = dist; } //-- Draw only "good" matches (i.e. whose distance is less than 3*min_dist ) std::vector<DMatch> good_matches; for(size_t i = 0; i < (size_t)descriptors_object.rows; i++) { if (matches[i].distance < 2 * min_dist) { good_matches.push_back(matches[i]); } } if (good_matches.size() < 4) return; Mat img_matches; drawMatches(img_object, keypoints_object, img_scene, keypoints_scene, good_matches, img_matches, Scalar::all(-1), Scalar::all(-1), vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS); //-- Localize the object std::vector<Point2f> obj; std::vector<Point2f> scene; for (size_t i = 0; i < (size_t)good_matches.size(); i++) { //-- Get the keypoints from the good matches obj.push_back(keypoints_object[(size_t)good_matches[i].queryIdx].pt); scene.push_back(keypoints_scene[(size_t)good_matches[i].trainIdx].pt); } vector<uchar> mask; Mat H = findHomography(obj, scene, CV_RANSAC, 3, mask); //-- Get the corners from the image_1 (the object to be "detected") std::vector<Point2f> obj_corners(4); obj_corners[0] = cvPoint(0, 0); obj_corners[1] = cvPoint(img_object.cols, 0); obj_corners[2] = cvPoint(img_object.cols, img_object.rows); obj_corners[3] = cvPoint(0, img_object.rows); std::vector<Point2f> scene_corners(4); perspectiveTransform(obj_corners, scene_corners, H); //-- Draw lines between the corners (the mapped object in the scene - image_2 ) line(img_matches, scene_corners[0] + Point2f(img_object.cols, 0), scene_corners[1] + Point2f(img_object.cols, 0), Scalar(0, 255, 0), 4); line(img_matches, scene_corners[1] + Point2f(img_object.cols, 0), scene_corners[2] + Point2f(img_object.cols, 0), Scalar(0, 255, 0), 4); line(img_matches, scene_corners[2] + Point2f(img_object.cols, 0), scene_corners[3] + Point2f(img_object.cols, 0), Scalar(0, 255, 0), 4); line(img_matches, scene_corners[3] + Point2f(img_object.cols, 0), scene_corners[0] + Point2f(img_object.cols, 0), Scalar(0, 255, 0), 4); ``` UPDATE: Here is the working solution as @mikesapi proposed: ``` ... //-- Draw only "good" matches (i.e. whose distance is less than 3*min_dist ) std::vector<DMatch> good_matches; double good_matches_sum = 0.0; for (size_t i = 0; i < matches.size(); i++ ) { if( matches[i].distance < max(2*min_dist, 0.02) ) { good_matches.push_back(matches[i]); good_matches_sum += matches[i].distance; } } double score = (double)good_matches_sum / (double)good_matches.size(); if (score < 0.18) { // have match } else { // dont have match } ... ```