Method of detecting and describing features from an intensity image

Patent metadata

Field	Value
Publication number	US-9679384-B2
Application number	US-201114240978-A
Country	US
Kind code	B2
Filing date	Aug 31, 2011
Priority date	Aug 31, 2011
Publication date	Jun 13, 2017
Grant date	Jun 13, 2017

Abstract

The invention provides methods of detecting and describing features from an intensity image. In one of several aspects, the method comprises the steps of providing an intensity image captured by a capturing device, providing a method for determining a depth of at least one element in the intensity image, in a feature detection process detecting at least one feature in the intensity image, wherein the feature detection is performed by processing image intensity information of the intensity image at a scale which depends on the depth of at least one element in the intensity image, and providing a feature descriptor of the at least one detected feature. For example, the feature descriptor contains at least one first parameter based on information provided by the intensity image and at least one second parameter which is indicative of the scale.

First claim

What is claimed is:

1. A method of detecting and describing features from an intensity image, comprising:
obtaining a 2D intensity image captured by a capturing device;
determining a depth of at least one element in the 2D intensity image based on light reflected from the at least one element;
scaling a size of a support region covering a portion of the 2D intensity image according to the determined depth;
detecting a feature in the 2D intensity image based on image intensity information of the support region; and
providing a feature descriptor of the detected feature,
wherein the scale at which the feature is detected depends on a depth sample for the support region.

2. The method according to claim 1 , wherein the feature descriptor contains at least one first parameter based on information provided by the 2D intensity image and at least one second parameter which is indicative of the scale of the size of the support region.

3. The method according to claim 1 , wherein
the feature descriptor describes the detected feature based on information provided by the 2D intensity image in a support region adjacent to the detected feature.

4. The method according to claim 1 , further comprising:
determining that the at least one element in the 2D intensity image belongs to the detected feature based on intensity values in the 2D intensity image.

5. The method according to claim 4 , wherein determining that at least one element belongs to the detected feature comprises performing feature detection at a scale that depends inversely proportional to the depth of the at least one element in the 2D intensity image.

6. The method according to claim 1 , wherein the scale at which the feature is detected corresponds to a physical size of the feature.

7. The method according to claim 1 , wherein the support region is scaled inversely proportional to the depth of the at least one element in the 2D intensity image.

8. The method according to claim 1 , further comprising: generating respective representations of the 2D intensity image for a plurality of scales, and
detecting the feature in the 2D intensity image at the respective scales.

9. The method according to claim 1 , wherein supporting points in the intensity image are used for providing the feature descriptor which comprise points specified by a geometry surrounding one of the points which has been identified in the feature detection process as being a part of the detected feature,
wherein the geometry varies according to the depth of the one of the points.

10. The method according to claim 1 , wherein the scale of the size of the support region is a global setting and the feature descriptor does not contain a second parameter indicative of the scale.

11. The method according to claim 1 , wherein determining a depth of at least one element in the 2D intensity image is based on an optical focus of the capturing device.

12. The method according to claim 1 , wherein determining a depth of at least one element in the 2D intensity image comprises generating depth samples of elements in the intensity image by extracting features of the 2D intensity image and at least one further 2D intensity image and matching the depth samples using the epipolar geometry of a stereo-camera pair capturing the intensity image and the at least one further 2D intensity image.

13. The method according to claim 1 , wherein the depth of the at least one element in the 2D intensity image is estimated using visual search algorithms to initially compare different distances.

14. The method according to claim 1 , further comprising:
obtaining a measurement of a position and orientation of the capturing device in a global coordinate system;
determining a pose of the capturing device from the measurement; and
generating a 3D model of an environment;
wherein the pose is used in combination with the 3D model to compute the depth of the at least one element.

15. The method according to claim 14 , further comprising tracking the capturing device with respect to an object of the intensity image captured by the capturing device using the 3D model of the environment.

16. A method of detecting and describing features from an intensity image, comprising:
obtaining a 2D intensity image captured by a capturing device;
determining a depth of at least one element in the 2D intensity image based on light reflected from the at least one element;
detecting at least one feature in the intensity image based on image intensity information provided by the 2D intensity image;
determining a depth of at least one element in the 2D intensity image, wherein the at least one element is part of the detected feature;
generating a support region surrounding the detected feature at a size scaled according to the determined depth; and
providing a feature descriptor of the at least one detected feature, the feature descriptor containing at least one first parameter based on information provided by the 2D intensity image, and at least one second parameter indicative of a combination of a depth of at least one element and a size of the support region,
wherein the second parameter is indicative of a product of the size of the support region and the depth of the at least one element in the 2D intensity image.

17. The method according to claim 16 , wherein the second parameter is used as a basis for a selection step in a subsequent feature matching process in which features of another 2D intensity image are considered as possible matches for the detected feature that have a feature descriptor including a parameter equivalent to the second parameter.

18. The method according to claim 16 , wherein the second parameter is invariant to a distance of the detected feature to the capturing device.

19. A non-transitory computer readable medium comprising software code which, when executed by one or more processors, causes the one or more processors to:
obtain a 2D intensity image captured by a capturing device;
determine a depth of at least one element in the 2D intensity image based on light reflected from the at least one element;
scale a size of a support region covering a portion of the 2D intensity image according to the determined depth;
detect at least one feature in the 2D intensity image based on image intensity information of the support region; and
provide a feature descriptor of the at least one detected feature,
wherein the scale at which the feature is detected depends on a depth sample for the support region.

20. The non-transitory computer readable medium of claim 19 , wherein the feature descriptor contains at least one first parameter based on information provided by the 2D intensity image and at least one second parameter which is indicative of the scale of the size of the support region.

21. The non-transitory computer readable medium of claim 19 , wherein the feature descriptor describes the detected feature based on information provided by the 2D intensity image in a support region surrounding the detected feature.

22. The non-transitory computer readable medium of claim 19 , further comprising software code configured to cause the one or more processors to determine that the at least one element in the 2D intensity image belongs to the detected feature based on intensity values in the 2D intensity image.

23. The non-transitory computer readable medium of claim 22 , wherein the software code configured to cause the one or more processors to de

Assignees

• Kurz Daniel
• Meier Peter
• Apple Inc

Inventors

• Kurz Daniel
• Meier Peter

Classifications

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06V10/32

  Normalisation of the pattern dimensions · CPC title

• G06V10/255

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title

• G06T7/73

  using feature-based methods · CPC title