Feature detection - Small item in a large picture

computer-vision, opencv

Solution

Based on limited experience, I have three recommendations for anyone else who runs into this type of issue.

1) Experiment with FindObject by Mathieu Labbé

This is a very nice tool that has helped me to experiment quickly to find good combinations of settings for feature detection / description. Just load your icon as an object and load your sample large images as scenes. Then tweak until you get reliable results for your application. As a bonus, he recently added the BRISK and FREAK non-patent-encumbered feature detection/description methods.

2) Get a realistic resolution

The resolution is so different for your icon object and actual icons in the large scenes that you are searching. In my understanding, the scale-invariance of these methods is actually pretty limited. Check out the excellent comparisons done by Ievgen Khvedchenia. You might get better results by resizing the image to the middle of the range that you expect it to be at.

3) Make the icon sample image more realistic (for example, blurred)

Related to #2, I have found that having a very sharp sample image when searching in a more realistic scene doesn't work well. I apply a gaussian to my sharp samples to make them more similar to what I expect to find. There is an example below. The strange formula just ensures that the kernel dimensions are odd numbers as required.

def proportional_gaussian(image):
    kernel_proportion = 0.005
    kernel_w = int(2.0 * round((image.shape[1]*kernel_proportion +1)/2.0)-1)
    kernel_h = int(2.0 * round((image.shape[0]kernel_proportion +1)/2.0)-1)
    return cv2.GaussianBlur(image, (kernel_w, kernel_h), 0)

Hope that helps someone.

Problem

Assume you have two images. In one you have a small icon (like less than 300X300 pixels). The second is a very large one, and in within you have one (or multiple) smaller instances of the icon (of course at different scale, rotation). The task at hand is to find the instances of the icon in the big image. How would you guys approach this? I've tried to use a feature based object detection, by using the OpenCV library, however for crowded big images (containing many feature points), the matching is inconclusive. I've tried the SURF/OBJ feature extractors with the BRUTE/FLAN matching algorithms. From my experience it seems like the matching takes no note of the geometrical relations between the feature points on the two images. As a visualization help I've attached an instance of the feature points of two example images. And here's a harder instance of the task. I've highlighted the icon in the big image.

Original source