Systems and methods for determining feature point motion

Patent metadata

Field	Value
Publication number	US-10284875-B2
Application number	US-201615231370-A
Country	US
Kind code	B2
Filing date	Aug 8, 2016
Priority date	Aug 8, 2016
Publication date	May 7, 2019
Grant date	May 7, 2019

Abstract

A method performed by an electronic device is described. The method includes obtaining a motion vector map based on at least two images. The motion vector map has fewer motion vectors than a number of pixels in each of the at least two images. The method also includes obtaining a feature point from one of the at least two images. The method further includes performing a matching operation between a template associated with the feature point and at least one search space based on the motion vector map. The method additionally includes determining a motion vector corresponding to the feature point based on the matching operation.

First claim

What is claimed is:

1. A method performed by an electronic device, comprising:
obtaining a motion vector map based on at least two images, the motion vector map having fewer motion vectors than a number of pixels in each of the at least two images;
obtaining a feature point from one of the at least two images, wherein the feature point is a corner or keypoint;
determining at least one search space based on a selected motion vector corresponding to an area of the motion vector map in response to determining that the feature point is included in the area, wherein a center point of the area corresponding to the selected motion vector is different from the feature point and the selected motion vector is applied to the feature point to determine the at least one search space;
performing a matching operation between a template associated with the feature point and the at least one search space based on the motion vector map; and
determining a motion vector corresponding to the feature point based on the matching operation.

2. The method of claim 1 , further comprising:
determining the template based on the feature point, wherein the template comprises a set of pixels including the feature point.

3. The method of claim 1 , wherein the motion vector map is determined by video encoder hardware.

4. The method of claim 1 , wherein the motion vector map is determined based on a first type of correlation, and wherein the motion vector corresponding to the feature point is determined based on a second type of correlation that is different from the first type of correlation.

5. The method of claim 4 , wherein the second type of correlation is a more accurate and computationally intensive type of correlation than the first type of correlation.

6. The method of claim 4 , wherein the first type of correlation is a sum of absolution differences (SAD) and the second type of correlation is a normalized cross correlation (NCC).

7. The method of claim 1 , wherein the matching operation is performed by correlation engine circuitry and determining the motion vector is performed by a processor that is separate from the correlation engine circuitry.

8. The method of claim 1 , further comprising tracking an object based on the motion vector.

9. An electronic device, comprising:
a motion engine, implemented in at least one of one or more hardware units, configured to obtain a motion vector map based on at least two images, the motion vector map having fewer motion vectors than a number of pixels in each of the at least two images;
a feature point obtainer, implemented in at least one of the one or more hardware units, configured to obtain a feature point from one of the at least two images, wherein the feature point is a corner or keypoint;
a search space determiner, implemented in at least one of the one or more hardware units, configured to determine at least one search space based on a selected motion vector corresponding to an area of the motion vector map in response to determining that the feature point is included in the area, wherein a center point of the area corresponding to the selected motion vector is different from the feature point and the selected motion vector is applied to the feature point to determine the at least one search space;
a matching engine, implemented in at least one of the one or more hardware units, configured to perform a matching operation between a template associated with the feature point and the at least one search space based on the motion vector map; and
a feature point motion vector determiner, implemented in at least one of the one or more hardware units, configured to determine a motion vector corresponding to the feature point based on the matching operation.

10. The electronic device of claim 9 , further comprising:
a template determiner, implemented in at least one of the one or more hardware units, configured to determine the template based on the feature point, wherein the template comprises a set of pixels including the feature point.

11. The electronic device of claim 9 , wherein the motion vector map is determined by the motion engine in video encoder hardware.

12. The electronic device of claim 9 , wherein the motion vector map is determined based on a first type of correlation, and wherein the motion vector corresponding to the feature point is determined based on a second type of correlation that is different from the first type of correlation.

13. The electronic device of claim 12 , wherein the second type of correlation is a more accurate and computationally intensive type of correlation than the first type of correlation.

14. The electronic device of claim 12 , wherein the first type of correlation is a sum of absolution differences (SAD) and the second type of correlation is a normalized cross correlation (NCC).

15. The electronic device of claim 9 , wherein the matching operation is performed by correlation engine circuitry and determining the motion vector is performed by a processor that is separate from the correlation engine circuitry.

16. The electronic device of claim 9 , further comprising an object tracker, implemented in at least one of the one or more hardware units, configured to track an object based on the motion vector.

17. An apparatus, comprising:
means for obtaining a motion vector map based on at least two images, the motion vector map having fewer motion vectors than a number of pixels in each of the at least two images;
means for obtaining a feature point from one of the at least two images, wherein the feature point is a corner or keypoint;
means for determining at least one search space based on a selected motion vector corresponding to an area of the motion vector map in response to determining that the feature point is included in the area, wherein a center point of the area corresponding to the selected motion vector is different from the feature point and the selected motion vector is applied to the feature point to determine the at least one search space;
means for performing a matching operation between a template associated with the feature point and the at least one search space based on the motion vector map; and
means for determining a motion vector corresponding to the feature point based on the matching operation.

18. The apparatus of claim 17 , further comprising:
means for determining the template based on the feature point, wherein the template comprises a set of pixels including the feature point.

19. The apparatus of claim 17 , wherein the motion vector map is determined by video encoder hardware.

20. The apparatus of claim 17 , wherein the motion vector map is determined based on a first type of correlation, and wherein the motion vector corresponding to the feature point is determined based on a second type of correlation that is different from the first type of correlation.

21. The apparatus of claim 20 , wherein the second type of correlation is a more accurate and computationally intensive type of correlation than the first type of correlation.

22. The apparatus of claim 20 , wherein the first type of correlation is a sum of absolution differences (SAD) and the second type of correlation is a normalized cross correlation (NCC).

23. The apparatus of claim 17 , wherein the means for performing a matching operation is a correlation engine and the means for determining the motion vector is a processor that is separate from the correlation engine.

24. A

Assignees

• Qualcomm Inc

Inventors

• Srinivasan Sujith
• Dane Gokce

Classifications

• H04N19/43

  Hardware specially adapted for motion estimation or compensation · CPC title

• H04N19/533

  Motion estimation using multistep search, e.g. two-dimensional [2D]-log search or one-at-a-time search [OTS] · CPC title

• H04N19/54

  using feature points or meshes · CPC title

• H04N19/56 Primary

  Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title