OpenCV Sift/Surf/Orb : drawMatch function is not working well
c++, opencv, point-of-interest
Solution
For somebody like me who searched for this but couldn't find the solution.
Assertion failed (i2 >= 0 && i2 < static_cast(keypoints2.size()))
This means that assertion failed due to i2 being less than 0 or i2 being less than the keypoints2 size. But what is i2?
From the link that rbaleksandar provided in a comment
int i2 = matches1to2[m].trainIdx;
trainIdx here is an index in keypoints2. The check i2 < static_cast(keypoints2.size()) makes sure that the index is less than keypoints2.size().
For me it occured because I discarded some keypoints before calling drawMatches but after the descriptors were computed i.e. DescriptorExtractor#compute was called. This meant that the drawMatches referred to old keypoints through the descriptors while I changed those keypoints. The end result was that some keypoints had large idx but the keypoints size was small hence the error.
Problem
I use Sift/Surf and ORB but sometimes i have a problem with the drawMatch function. Here the error : OpenCV Error: Assertion failed (i2 >= 0 && i2 < static_cast(keypoints2.size())) in drawMatches, file /home/opencv-2.4.6.1/modules/features2d/src/draw.cpp, line 208 terminate called after throwing an instance of 'cv::Exception' what(): /home/opencv-2.4.6.1/modules/features2d/src/draw.cpp:208: error: (-215) i2 >= 0 && i2 < static_cast(keypoints2.size()) in function drawMatche The code : ``` drawMatchPoints(img1,keypoints_img1,img2,keypoints_img2,matches); ``` I tried to invert img 1,keypoints_img1 with img2 and keypoints_img2 like that : ``` drawMatchPoints(img2,keypoints_img2,img1,keypoints_img1,matches); ``` Corresponding to my function who is doing an homography : ``` void drawMatchPoints(cv::Mat image1,std::vector<KeyPoint> keypoints_img1, cv::Mat image2,std::vector<KeyPoint> keypoints_img2,std::vector<cv::DMatch> matches){ cv::Mat img_matches; drawMatches( image1, keypoints_img1, image2, keypoints_img2, matches, img_matches, Scalar::all(-1), Scalar::all(-1), vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS ); std::cout << "Number of good matching " << (int)matches.size() << "\n" << endl; //-- Localize the object std::vector<Point2f> obj; std::vector<Point2f> scene; for( int i = 0; i < matches.size(); i++ ) { //-- Get the keypoints from the good matches obj.push_back( keypoints_img1[ matches[i].queryIdx ].pt ); scene.push_back( keypoints_img2[matches[i].trainIdx ].pt ); } Mat H = findHomography( obj, scene, CV_RANSAC ); std::cout << "Size of homography " << *H.size << std::endl ; //-- Get the corners from the image_1 ( the object to be "detected" ) std::vector<Point2f> obj_corners(4); obj_corners[0] = cvPoint(0,0); obj_corners[1] = cvPoint( image1.cols, 0 ); obj_corners[2] = cvPoint( image1.cols, image1.rows ); obj_corners[3] = cvPoint( 0, image1.rows ); std::vector<Point2f> scene_corners(4); perspectiveTransform( obj_corners, scene_corners, H); //-- Draw lines between the corners (the mapped object in the scene - image_2 ) line( img_matches, scene_corners[0] + Point2f( image1.cols, 0), scene_corners[1] + Point2f( image1.cols, 0), Scalar(0, 255, 0), 4 ); line( img_matches, scene_corners[1] + Point2f( image1.cols, 0), scene_corners[2] + Point2f( image1.cols, 0), Scalar( 0, 255, 0), 4 ); line( img_matches, scene_corners[2] + Point2f( image1.cols, 0), scene_corners[3] + Point2f( image1.cols, 0), Scalar( 0, 255, 0), 4 ); line( img_matches, scene_corners[3] + Point2f( image1.cols, 0), scene_corners[0] + Point2f( image1.cols, 0), Scalar( 0, 255, 0), 4 ); //-- Show detected matches cv::imshow( "Good Matches & Object detection", img_matches ); cv::waitKey(5000); ``` } But i have still the error ! I have noticed that the error happened when the size of my keypoints_img1 is lower than the size of my keypoints_img2 : Size keyPoint1 : 244 - Size keyPoint2 : 400 So if i invert the loading of my two picture, that works but i can't now in advance if my first picture will have more keypoints that my second picture... My code (the most important step) in order to create the features : ``` init_Sift(400,5,0.04,25,1.6); void init_Sift(int nf,int nOctaveL,double contrastThresh, double edgeThresh,double sigma){ this->nfeatureSift=nf; this->nOctaveLayerSift=nOctaveL; this->contrastThresholdSift=contrastThresh; this->edgeThresholdSift=edgeThresh; this->sigmaSift=sigma;} cv::FeatureDetector* detector=new SiftFeatureDetector(nfeatureSift,nOctaveLayerSift,contrastThresholdSift,edgeThresholdSift,sigmaSift); cv::DescriptorExtractor* extractor=new SiftDescriptorExtractor extractor->compute( image, keypoints, descriptors ); ``` The matching part : ``` std::cout << "Type of matcher : " << type_of_matcher << std::endl; if (type_of_matcher=="FLANN" || type_of_matcher=="BF"){ std::vector<KeyPoint> keypoints_img1 = keyfeatures.compute_Keypoints(img1); std::vector<KeyPoint> keypoints_img2 = keyfeatures.compute_Keypoints(img2); cv::Mat descriptor_img1 = keyfeatures.compute_Descriptors(img1); cv::Mat descriptor_img2 = keyfeatures.compute_Descriptors(img2); std::cout << "Size keyPoint1 " << keypoints_img1.size() << "\n" << std::endl; std::cout << "Size keyPoint2 " << keypoints_img2.size() << "\n" << std::endl; //Flann with sift or surf if (type_of_matcher=="FLANN"){ Debug::info("USING Matcher FLANN"); fLmatcher.match(descriptor_img1,descriptor_img2,matches); double max_dist = 0; double min_dist = 100; //-- Quick calculation of max and min distances between keypoints for( int i = 0; i < descriptor_img1.rows; i++ ){ double dist = matches[i].distance; if( dist < min_dist ) min_dist = dist; if( dist > max_dist ) max_dist = dist; } std::vector< DMatch > good_matches; for( int i = 0; i < descriptor_img1.rows; i++ ) { if( matches[i].distance <= max(2*min_dist, 0.02) ) { good_matches.push_back( matches[i]); } } std::cout << "Size of good match : " << (int)good_matches.size() << std::endl; //-- Draw only "good" matches if (!good_matches.empty()){ drawMatchPoints(img1,keypoints_img1,img2,keypoints_img2,good_matches); } else { Debug::error("Flann Matcher : Pas de match"); cv::Mat img_matches; drawMatches( img1, keypoints_img1, img2, keypoints_img2, matches, img_matches, Scalar::all(-1), Scalar::all(-1), vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS ); cv::imshow( "No match", img_matches ); cv::waitKey(5000); } } //BruteForce with sift or surf else if (type_of_matcher=="BF"){ Debug::info("USING Matcher Brute Force"); bFmatcher.match(descriptor_img1,descriptor_img2,matches); if (!matches.empty()){ std::nth_element(matches.begin(),//Initial position matches.begin()+24, //Position of the sorted element matches.end());//End position matches.erase(matches.begin()+25,matches.end()); drawMatchPoints(img1,keypoints_img1,img2,keypoints_img2,matches); //drawMatchPoints(img2,keypoints_img2,img1,keypoints_img1,matches); } else { Debug::error("Brute Force matcher : Pas de match"); cv::Mat img_matches; drawMatches( img1, keypoints_img1, img2, keypoints_img2, matches, img_matches, Scalar::all(-1), Scalar::all(-1), vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS ); cv::imshow( "No match", img_matches ); cv::waitKey(5000); } } ``` Do you have any suggestions or advises ? EDIT : i solved my problem. I had a c++ problem, because i had two classes . One about matching and another about finding keyFeature. I have wrote on my .h std::vector and the same for descriptors. ``` class keyFeatures{ public: ... std::vector<keyPoint> keypoints; ... ``` I delete this attribute and i did a function that take in argument std::vector keypoints ``` cv::Mat descriptor_img1 = keyfeatures.compute_Descriptors(img1,keypoints_img1); ``` instead of ``` cv::Mat descriptor_img1 = keyfeatures.compute_Descriptors(img1); ``` I think there was a conflict when i did the matching... But i don't know why i shouldn't have to write it on my .h and do a local parameter on my function. Thank you !