Feature Matching

Feature matching  – used to find corresponding points or features between two images,

Essential for object recognition, image stitching, 3D reconstruction, and motion tracking.

Feature Detection

Features (key points) are detected in both images. Common feature detectors are:

• SIFT (Scale – Invariant Feature Transform)
• SURF (Speeded – Up Robust Features)
• ORB (Oriented FAST and Rotated BRIEF)

Feature Descriptor

After detecting keypoints, a descriptor is generated for each keypoint

Descriptors represent the neighborhood around the keypoints

Example: 

• SIFT provides 128 dimensional descriptors
• ORB provides binary descriptors (32 bytes – 256 bits)

Feature Matching

Once the features are detected and described, the next step is to match between images. Common methods include:

Brute Force matcher

• Compares every descriptor in one image with every descriptor in other image and selects the best match based on a chosen distance metric

FLANN

• Fast Library for Approximate Nearest Neighbors

ORB ( Oriented Fast and Rotated Brief)

• FAST Corner Detection (Features from Accelerated Segment Test)
• Orientation Assignment
• BRIEF Descriptor

Orientation Calculation:

• After detecting a corner using FAST, ORB calculates the direction (orientation) of the corner.
• It does this by finding the intensity-weighted centroid of a patch around the corner.
• The direction from the corner to this centroid gives the orientation θ, making the feature rotation-invariant.

BRIEF Descriptor Calculation:

• ORB uses the BRIEF descriptor, which compares the brightness of pairs of pixels around the corner.
• ORB rotates the BRIEF pattern according to the corner’s orientation θ, ensuring the descriptor is consistent even if the image is rotated.

Feature Matching:

• Once the descriptors are calculated for corners in two images, ORB matches them using the Hamming distance (a method for comparing binary strings).
• Corners with similar descriptors in two images are matched as the same points.

Features from Accelerated Segment Test

Step 1: Select a pixel p in the image which is to be identified as an interest point or not. Let its intensity be I_p.

Step 2: Select appropriate threshold value t.

Step 3: Consider a circle of 16 pixels around the pixel under test.

Step 4: a pixel ‘p’ is considered a corner if there is a set of ‘n’ contiguous pixels in the circle of 16 pixels around it, which are either:

• Brighter than I_p+T  (where T is the threshold).
• Darker than I_p – T ‘n’ is chosen to be 12, meaning at least 12 contiguous pixels around ‘p’ should be consistently brighter or darker than the threshold condition for ‘p’ to be classified as a corner.

Step 5: High Speed Optimization Test

• To quickly eliminate non-corner pixels, the algorithm first performs a high-speed test using only four specific pixels in the circle: pixels at positions 1, 5, 9, and 13 (in the 16-pixel circle).
• Check pixels at positions 1 and 9. If both of these pixels are either brighter than or darker then go to next step.
• Check pixels at positions 5 and 13. If both of these pixels also meet the same condition, then there’s a high chance that the pixel p is a corner.
• If at least three of these four pixels meet the condition either brighter than or darker than, only then is the pixel ‘p’ a potential corner.
• If the condition is not met, the pixel is not a corner, and further tests are skipped.
• If neither of these is the case, then p cannot be a corner.
• The full segment test criterion can then be applied to the passed candidates by examining all pixels in the circle

Feature Matching – ORB

For each pair of points (p₁,p₂) relative to the keypoint location:

• Compare the intensity values of the two points.
• Construct the binary descriptor based on these comparisons

The Hamming distance helps determine how similar the two descriptors are:

• If the Hamming distance is low, it indicates that the key points from both images are likely similar and correspond to the same feature in the scene.
• A higher Hamming distance suggests that the key points are less similar.

keypoints, descriptors = cv2.ORB.detectAndCompute(image, mask) 

image:

• This is the input image where keypoints need to be detected. It should be a single-channel image (grayscale).

mask:

• This is an optional parameter.
• Mask image, which is a binary image where non-zero pixels indicate the area of interest for keypoint detection.
• If None is passed (or if this parameter is omitted), the whole image is considered for detecting keypoints.

keypoints:

• A list of detected keypoints. Each keypoint is represented as an instance of cv2.KeyPoint, which contains information like the keypoint’s position, size, angle, response, and more.

descriptors:

• A NumPy array of shape (number_of_keypoints, descriptor_size), where descriptor_size is typically 32 bytes (256 bits) for ORB descriptors. If no keypoints are found, this will return None.

bf = cv2.BFMatcher(normType, crossCheck)

normType:

• This specifies the norm used to compute the distance between descriptors. For binary descriptors
  •                   cv2.NORM_HAMMING: This calculates the Hamming distance, which is appropriate for binary descriptors.

crossCheck:

• This is a boolean parameter.
• If set to True, the matcher will only consider matches that are consistent in both directions (mutually best matches).
• If set to False, the matcher will find the nearest matches without this validation.

matches = bf.match(des1, des2) 

des1: The first set of descriptors to be matched (e.g., from the first image).

des2: The second set of descriptors to be matched (e.g., from the second image).

matches:

• A list of DMatch objects, where each DMatch contains information about the matched keypoints, including:
• queryIdx: Index of the descriptor in the first descriptor set (des1).
• trainIdx: Index of the descriptor in the second descriptor set (des2).
• distance: The distance between the matched descriptors.

Views: 1