Efficient descriptor extraction over multiple levels of an image scale space

Patent metadata

Field	Value
Publication number	US-9530073-B2
Application number	US-201113090180-A
Country	US
Kind code	B2
Filing date	Apr 19, 2011
Priority date	Apr 20, 2010
Publication date	Dec 27, 2016
Grant date	Dec 27, 2016

Abstract

A local feature descriptor for a point in an image is generated over multiple levels of an image scale space. The image is gradually smoothened to obtain a plurality of scale spaces. A point may be identified as the point of interest within a first scale space from the plurality of scale spaces. A plurality of image derivatives is obtained for each of the plurality of scale spaces. A plurality of orientation maps is obtained (from the plurality of image derivatives) for each scale space in the plurality of scale spaces. Each of the plurality of orientation maps is then smoothened (e.g., convolved) to obtain a corresponding plurality of smoothed orientation maps. Therefore, a local feature descriptor for the point may be generated by sparsely sampling a plurality of smoothed orientation maps corresponding to two or more scale spaces from the plurality of scale spaces.

First claim

What is claimed is:

1. A method for generating a local feature descriptor for an image, comprising:
identifying a point within a first scale space from a plurality of scale spaces for an image;
obtaining a plurality of image derivatives for each of the plurality of scale spaces;
obtaining a plurality of orientation maps for each scale space in the plurality of scale spaces, where each of the plurality of orientation maps is obtained from non-negative values of a corresponding image derivative;
for each scale space in the plurality of scale spaces, smoothing each of the plurality of orientation maps to obtain a corresponding plurality of smoothed orientation maps; and
sparsely sampling a plurality of smoothed orientation maps corresponding to two or more scale spaces from the plurality of scale spaces to generate a local feature descriptor for the point
wherein each of the smoothed orientation maps that is sparsely sampled is derived using a different orientation map than each of the other smoothed orientation maps that is sparsely sampled.

2. The method of claim 1 , further comprising gradually smoothing the image to obtain the plurality of scale spaces.

3. The method of claim 1 , wherein said smoothing comprises:
smoothing each of the plurality of orientation maps for the first scale space by a first smoothing coefficient to generate the corresponding plurality of smoothed orientation maps; and
smoothing each of the plurality of orientation maps for a second scale space in the plurality of scale spaces by the first smoothing coefficient to generate the corresponding plurality of smoothed orientation maps,
wherein the second scale space is different than the first scale space.

4. The method of claim 1 , wherein the point is a sample point from a subset of locations within the plurality of scale spaces, and
wherein the subset of locations is selected based on a pattern that represents an object to be detected.

5. The method of claim 1 , wherein the point is a sample point from a subset of locations within the plurality of scale spaces, and
wherein the subset of locations is selected based on identified keypoints within the image, wherein a keypoint is a point that has been identified as being robust to changes in imaging conditions.

6. The method of claim 1 , wherein the two or more scale spaces include the first scale space and one or more additional scale spaces of lower resolution than the first scale space.

7. The method of claim 1 , wherein the local feature descriptor has a kernel pooling configuration defined by spatial pooling of sample points distributed over a center of the point.

8. The method of claim 1 , wherein sparsely sampling a plurality of smoothed orientation maps includes
sampling a first plurality of points on a first smoothed orientation map, the first plurality of points arranged in a first ring concentric with a location of the point; and
sampling a second plurality of points on a second smoothed orientation map, the second plurality of points arranged in a second ring concentric with the location of the point, the second smoothed orientation map corresponding to a second scale space of lower resolution than the first scale space.

9. The method of claim 8 , wherein sparsely sampling a plurality of smoothed orientation maps further includes
sampling a third plurality of points on a third smoothed orientation map, the third plurality of points arranged in a third ring concentric with the location of the point, the third smoothed orientation map corresponding to a third scale space of lower resolution than the first scale space.

10. The method of claim 9 , wherein the second ring has a second radius greater than a first radius for the first ring, and the third ring has a third radius greater than the second radius for the second ring.

11. The method of claim 1 , wherein the plurality of orientation maps for each scale space include orientation maps for a plurality of different orientations.

12. The method of claim 1 , further comprising:
building a plurality of histograms of oriented gradients from the sparse sampling of the plurality of smoothed orientation maps, wherein the local feature descriptor comprises the plurality of histograms.

13. The method of claim 1 , wherein said method includes, subsequent to said identifying and prior to said sparsely sampling, selecting a scale space to be among the two or more scale spaces, based on a smoothing coefficient of the scale space.

14. An image processing device, comprising:
an input interface adapted to obtain an image;
a storage device to store local feature descriptors for one or more images;
a hardware processing circuit coupled to the input interface and the storage device, the hardware processing circuit adapted to:
identify a point within a first scale space from a plurality of scale spaces for an image;
obtain a plurality of image derivatives for each of the plurality of scale spaces;
obtain a plurality of orientation maps for each scale space in the plurality of scale spaces, where each of the plurality of orientation maps is obtained from non-negative values of a corresponding image derivative;
for each scale space in the plurality of scale spaces, smooth each of the plurality of orientation maps to obtain a corresponding plurality of smoothed orientation maps; and
sparsely sample a plurality of smoothed orientation maps corresponding to two or more scale spaces from the plurality of scale spaces to generate a local feature descriptor for the point,

wherein each of the smoothed orientation maps that is sparsely sampled is derived using a different orientation map than each of the other smoothed orientation maps that is sparsely sampled.

15. The device of claim 14 , wherein the two or more scale spaces include the first scale space and one or more additional scale spaces of lower resolution than the first scale space.

16. The device of claim 14 , wherein the processing circuit is further adapted to gradually smooth the image to obtain the plurality of scale spaces.

17. The device of claim 14 , wherein the local feature descriptor has a kernel pooling configuration defined by spatial pooling of sample points distributed over a center on the point.

18. The device of claim 14 , wherein sparsely sampling a plurality of smoothed orientation maps includes
sampling a first plurality of points on a first smoothed orientation map, the first plurality of points arranged in a first ring concentric with a location of the point; and
sampling a second plurality of points on a second smoothed orientation map, the second plurality of points arranged in a second ring concentric with the location of the point, the second smoothed orientation map corresponding to a second scale space of lower resolution than the first scale space.

19. The device of claim 14 , wherein the processing circuit is further adapted to:
build a plurality of histograms of oriented gradients from the sparse sampling of the plurality of smoothed orientation maps, wherein the local feature descriptor comprises the plurality of histograms.

20. The device of claim 14 , wherein said hardware processing circuit is adapted to select, subsequent to said identifying and prior to said sparsely sampling, a scale space to be among the two or more scale spaces, based on a smoothing coefficient of the scale space.

21. An image processing device, comprising:
means for identifying a point within a first scale space from a plurality of scale spac

Assignees

• Hamsici Onur C
• Hong John H
• Reznik Yuriy
• Vaddadi Sundeep
• Lee Chong Uk
• Qualcomm Inc

Inventors

• Hamsici Onur C
• Hong John H
• Reznik Yuriy
• Vaddadi Sundeep
• Lee Chong Uk

Classifications

• G06V10/462 Primary

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06K9/4671 Primary

  Physics · mapped topic

• G06V10/40

  Extraction of image or video features · CPC title

• G06T7/00 Primary

  Image analysis · CPC title