Method and apparatus for detecting a salient point of a protuberant object

Patent metadata

Field	Value
Publication number	US-10867386-B2
Application number	US-201616098403-A
Country	US
Kind code	B2
Filing date	Jun 30, 2016
Priority date	Jun 30, 2016
Publication date	Dec 15, 2020
Grant date	Dec 15, 2020

Abstract

An image processing method and an apparatus ( 300 ) are provided, the method includes: obtaining a depth image of a protuberant object ( 210 ); selecting a plurality of test points placed on a circle around a pixel in the depth image as a center point of the circle; calculating a protuberance value of the center point based on a comparison between the depth value of the center point and the depth value of each of the selected test points ( 240 ); and determining one or more salient points of the protuberant object by using the protuberance value of each pixel in the depth image ( 250 ).

First claim

The invention claimed is:

1. An image processing method, comprising:
obtaining a depth image of a protuberant object;
selecting a plurality of test points placed on a circle around a pixel in the depth image as a center point of the circle;
calculating a protuberance value of the center point based on a comparison between the depth value of the center point and the depth value of each of the selected test points, wherein a coarse-to-fine strategy with two or more radii is applied to calculate the protuberance value of the center point; and
determining one or more salient points of the protuberant object by using the protuberance value of each pixel in the depth image.

2. The method of claim 1 , wherein a radius of the circle is determined based at least in part on the depth value of the center point.

3. The method of claim 1 , wherein the test points are symmetrically placed on the circle.

4. The method of claim 1 , the calculating comprising:
counting a number of positive comparison results associated with the selected test points, wherein a positive comparison result is generated if the depth value of one test point is larger than the depth value of the center point,
wherein the larger the number of positive comparison results is the larger the protuberance value of the center point is.

5. The method of claim 4 , the calculating further comprising:
subtracting the depth value of the center point from the depth value of each of the selected test points; and
counting a number of zero-crossings associated with the selected test points, wherein a zero crossing is generated if a sign of a subtracted result associated with one test point is different from a sign of a subtracted result associated with its neighboring test point,
wherein the smaller the number of zero-crossings is the larger the protuberance value of the center point is.

6. The method of claim 1 , wherein the coarse-to-fine strategy comprises voting the calculated protuberance value with a highest appearance frequency is voted as the protuberance value of the center point.

7. The method of claim 1 , wherein the coarse-to-fine strategy comprises voting the calculated protuberance values are averaged as the protuberance value of the center point.

8. The method of claim 1 , further comprising:
balancing the protuberance value of each pixel in the depth image with a corresponding depth value of the pixel.

9. The method of claim 8 , the balancing comprising:
normalizing the depth value of the center point; and
multiplying the protuberance value of the center point with the normalized depth value of the center point.

10. The method of claim 8 , further comprising:
applying a regression-based approach or a classification-based approach to the balanced protuberance value of each pixel in the depth image.

11. The method of claim 10 , wherein the regression-based approach or classification-based approach comprises one of the following approaches: Random Ferns, Random Decision Forests, Convolutional Neural Network (CNN), Support Vector Machine (SVM).

12. An image processing apparatus, comprising:
at least one processor; and
memory including instructions that, when executed by the at least one processor, cause the at least one processor to perform operations to:
obtain a depth image of a protuberant object;
select a plurality of test points placed on a circle around a pixel in the depth image as a center point of the circle, and calculate a protuberance value of the center point based on a comparison between the depth value of the center point and the depth value of each of the selected test points, wherein a coarse-to-fine strategy with two or more radii is applied to calculate the protuberance value of the center point; and
detect one or more salient points of the protuberant object by using the protuberance value of each pixel in the depth image.

13. The apparatus of claim 12 , the memory further comprising instructions that cause the at least one processor to determine a radius of the circle based at least in part on the depth value of the center point.

14. The apparatus of claim 12 , the memory further comprising instructions that cause the at least one processor to count a number of positive comparison results associated with the selected test points, wherein a positive comparison result is generated if the depth value of one test point is larger than the depth value of the center point,
wherein the larger the number of positive comparison results is the larger the protuberance value of the center point is.

15. The apparatus of claim 14 , the memory further comprising instructions that cause the at least one processor to:
subtract the depth value of the center point from the depth value of each of the selected test points; and
count a number of zero-crossings associated with the selected test points, wherein a zero crossing is generated if a sign of a subtracted result associated with one test point is different from a sign of a subtracted result associated with its neighboring test point,
wherein the smaller the number of zero-crossings is the larger the protuberance value of the center point is.

16. The apparatus of claim 12 , the memory further comprising instructions that cause the at least one processor to balance the protuberance value of each pixel in the depth image with a corresponding depth value of the pixel.

17. The apparatus of claim 16 , the instructions to perform the balancing further comprising instructions that cause the at least one processor to:
normalize the depth value of the center point; and
multiply the protuberance value of the center point with the normalized depth value of the center point.

18. The apparatus of claim 16 , the memory further comprising instructions that cause the at least one processor to apply a regression-based approach or a classification-based approach to the balanced protuberance value of each pixel in the depth image.

19. A computer system, comprising:
one or more processors; and
a memory storing computer-executable instructions that, when executed, cause the one or more processors to:
obtain a depth image of a protuberant object;
select a plurality of test points placed on a circle around a pixel in the depth image as a center point of the circle;
calculate a protuberance value of the center point based on a comparison between the depth value of the center point and the depth value of each of the selected test points, wherein a coarse-to-fine strategy with two or more radii is applied to calculate the protuberance value of the center point; and
determine one or more salient points of the protuberant object by using the protuberance value of each pixel in the depth image.

Assignees

• Microsoft Technology Licensing Llc
• Huo Qiang
• Ban Yuseok

Inventors

• Huo Qiang
• Ban Yuseok

Classifications

• G06T7/60 Primary

  Analysis of geometric attributes · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06V10/36

  Applying a local operator, i.e. means to operate on image points situated in the vicinity of a given point; Non-linear local filtering operations, e.g. median filtering · CPC title