Pattern radius adjustment for keypoint descriptor generation

Patent metadata

Field	Value
Publication number	US-11810266-B2
Application number	US-202117206852-A
Country	US
Kind code	B2
Filing date	Mar 19, 2021
Priority date	Mar 19, 2021
Publication date	Nov 7, 2023
Grant date	Nov 7, 2023

Abstract

Embodiments relate to generating keypoint descriptors of the keypoints using a sub-scale refinement and a sample pattern radius adjustment. An apparatus includes a sub-pixel refiner circuit and a keypoint descriptor generator circuit. The sub-pixel refiner circuit determines a keypoint scale value for a scale dimension of a keypoint in an image pyramid by performing an interpolation of response map (RM) pixel values of a pixel block of RM images defined around the keypoint. The keypoint descriptor generator circuit determines sample scales of the image pyramid based on the keypoint scale value and determines a radius value for each sample scale based on the keypoint scale value. The keypoint descriptor generator circuit samples patches of pixel values at the sample scales using the radius value for each sample scale to generate a keypoint descriptor of the keypoint.

First claim

What is claimed is:

1. An apparatus, comprising:
a sub-pixel refiner circuit configured to determine a keypoint scale value for a scale dimension of a keypoint in an image pyramid by performing an interpolation of response map (RM) pixel values of a pixel block of RM images defined around the keypoint, the RM images each corresponding with a pyramid image of the image pyramid; and
a keypoint descriptor generator circuit coupled to the sub-pixel refiner circuit, the keypoint descriptor generator circuit configured to:
determine sample scales of the image pyramid for the keypoint based on a comparison between the keypoint scale value and a keypoint scale where the keypoint is physically located;
determine, for each sample scale, a radius value based on the keypoint scale value, the radius value defining locations of sample points in a pyramid image at the sample scale;
determine, for each of the sample points of each sample scale, a patch of pixel values based on the radius value of the sample scale; and
generate a keypoint descriptor of the keypoint using the patches of the sample points of each sample scale.

2. The apparatus of claim 1 , further comprising:
a pyramid image generator circuit configured to generate the image pyramid from an input image; and
a RM generator circuit coupled to the pyramid image generator circuit and the sub-pixel refiner circuit, the RM generator circuit configured to generate the RM images using pyramid images.

3. The apparatus of claim 1 , wherein:
the scale dimension of the image pyramid defines different pyramid images, each pyramid image including a first planar dimension and a second planar dimension of pixel values that is perpendicular to the first planar dimension;
the sub-pixel refiner circuit is further configured to determine a first planar value for the first planar dimension and a second planar value for the second planar dimension of the keypoint in the pyramid image by performing the interpolation of the RM pixel values for the pixel block defined around a physical center point of the keypoint, the physical center point located at a first planar location along the first planar dimension, a second planar location along the second planar dimension, and the keypoint scale along the scale dimension; and
the radius value for each sample scale defines the locations of the sample points of the sample scale with respect to a refined center point defined by the first planar value and the second planar value of the keypoint.

4. The apparatus of claim 3 , wherein:
the first planar location, the second planar location, and the keypoint scale are integer values; and
at least one of the first planar value, the second planar value, or the keypoint scale value is a non-integer value.

5. The apparatus of claim 1 , wherein the radius value is different for different sample scales.

6. The apparatus of claim 1 , wherein the keypoint descriptor generator circuit configured to determine the sample scales includes the keypoint descriptor generator circuit being configured to:
determine whether the keypoint scale value is greater than or less than the keypoint scale; and
responsive to determining that the keypoint scale value is greater than the keypoint scale, determine the sample scales as including the keypoint scale and a plurality of higher scales.

7. The apparatus of claim 1 , wherein the keypoint descriptor generator circuit configured to determine the sample scales includes the keypoint descriptor generator circuit being configured to:
determine whether the keypoint scale value is greater than or less than the keypoint scale; and
responsive to determining that the keypoint scale value is less than the keypoint scale, determine the sample scales as including two instances of the keypoint scale using different radius values and a plurality of higher scales.

8. The apparatus of claim 1 , wherein the radius value of each sample scale is a function of the keypoint scale value if the sample scale is the keypoint scale or the keypoint scale value incremented by an integer value if the sample scale is a higher scale than the keypoint scale.

9. The apparatus of claim 1 , wherein the keypoint descriptor generator circuit is further configured to:
determine, for each of the sample points of each sample scale, an intensity value based on the patch of the sample point; and
determine comparison results of comparisons between the intensity values of pairs of the sample points, the keypoint descriptor of the keypoint including the comparison results.

10. The apparatus of claim 1 , wherein the locations of the sample points of each sample scale are further defined by a sample pattern and adjacent sample scales that use different sample patterns.

11. The apparatus of claim 1 , wherein the sample scales exclude scales lower than the keypoint scale.

12. A method, comprising:
determining, by a sub-pixel refiner circuit, a keypoint scale value for a scale dimension of a keypoint in an image pyramid by performing an interpolation of response map (RM) pixel values of a pixel block of RM images defined around the keypoint, the RM images each corresponding with a pyramid image of the image pyramid;
determining, by a keypoint descriptor generator circuit, sample scales of the image pyramid for the keypoint based on a comparison between the keypoint scale value and a keypoint scale where the keypoint is physically located;
determining, by the keypoint descriptor generator circuit and for each sample scale, a radius value based on the keypoint scale value, the radius value defining locations of sample points in a pyramid image at the sample scale;
determining, by the keypoint descriptor generator circuit and for each of the sample points of each sample scale, a patch of pixel values based on the radius value of the sample scale; and
generating, by the keypoint descriptor generator circuit, a keypoint descriptor of the keypoint using the patches of the sample points of each sample scale.

13. The method of claim 12 , wherein:
the scale dimension of the image pyramid defines different pyramid images, each pyramid image including a first planar dimension and a second planar dimension of pixel values that is perpendicular to the first planar dimension;
the method further includes, by the sub-pixel refiner circuit, determining a first planar value for the first planar dimension and a second planar value for the second planar dimension of the keypoint in the pyramid image by performing the interpolation of the RM pixel values for the pixel block defined around a physical center point of the keypoint, the physical center point located at a first planar location along the first planar dimension, a second planar location along the second planar dimension, and the keypoint scale along the scale dimension; and
the radius value for each sample scale defines the locations of the sample points of the sample scale with respect to a refined center point defined by the first planar value and the second planar value of the keypoint.

14. The method of claim 13 , wherein:
the first planar location, the second planar location, or the keypoint scale are integer values; and
at least one of the first planar value, the second planar value, and the keypoint scale value is a non-integer value.

15. The method of claim 12 , wherein the radius value is different for different sample scales.

16. The method of claim 12 , wherein determining the sample scales includes:
determining whether the keypoint scale value is greater than or less than the keypoint scale; and
responsive to determinin

Assignees

• Apple Inc

Inventors

• Wang Muge
• Metuki Assaf

Classifications

• G06T3/4038 Primary

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T3/4069

  by subpixel displacements · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06T15/005

  General purpose rendering architectures · CPC title

• G06V10/462 Primary

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title