Feature point identification in sparse optical flow based tracking in a computer vision system

Patent metadata

Field	Value
Publication number	US-11915431-B2
Application number	US-201916532658-A
Country	US
Kind code	B2
Filing date	Aug 6, 2019
Priority date	Dec 30, 2015
Publication date	Feb 27, 2024
Grant date	Feb 27, 2024

Abstract

A method for sparse optical flow based tracking in a computer vision system is provided that includes detecting feature points in a frame captured by a monocular camera in the computer vision system to generate a plurality of detected feature points, generating a binary image indicating locations of the detected feature points with a bit value of one, wherein all other locations in the binary image have a bit value of zero, generating another binary image indicating neighborhoods of currently tracked points, wherein locations of the neighborhoods in the binary image have a bit value of zero and all other locations in the binary image have a bit value of one, and performing a binary AND of the two binary images to generate another binary image, wherein locations in the binary image having a bit value of one indicate new feature points detected in the frame.

First claim

What is claimed is:

1. A method comprising:
receiving a first frame;
detecting a first set of feature points within the first frame;
generating a first image that indicates locations of the first set of feature points detected within the first frame, wherein the locations of the first set of feature points are indicated in the first image by a first binary value and a remainder of the first image is indicated by a second binary value;
generating a second image that indicates respective bounded neighborhoods each of which corresponds to a neighborhood surrounding one of a second set of feature points detected based on a second frame, wherein the respective bounded neighborhoods are indicated in the second image by the second binary value and a remainder of the second image is indicated by the first binary value; and
comparing the first image to the second image to generate a third image, wherein locations of a subset of the first set of feature points excluded from the respective bounded neighborhoods are indicated in the third image by a binary value different from a remainder of the third image.

2. The method of claim 1 , wherein the second frame is captured by a camera prior to the first frame.

3. The method of claim 1 further comprising generating a third set of feature points that includes the subset of the first set of feature points and the second set of feature points.

4. The method of claim 1 , further comprising determining a set of coordinates for each feature point of the subset of the first set of feature points based on the third image.

5. The method of claim 1 , wherein the first binary value is a binary one and the second binary value is a binary zero.

6. The method of claim 1 , wherein each of the bounded neighborhoods has a 3×3 pixel area or a 5×5 pixel area.

7. The method of claim 1 ,
wherein dimensions of the first image are equal to dimensions of the first frame, and
wherein dimensions of the second image are equal to the dimensions of the first frame.

8. A computer vision system comprising:
a detection circuit configured to detect a first set of feature points in a first frame; and
a feature point identification circuit coupled to the detection circuit and configured to:
receive the first set of feature points from the detection circuit;
generate a first image that indicates locations of the first set of feature points, wherein the locations of the first set of feature points are indicated in the first image by a first binary value and a remainder of the first image is indicated by a second binary value;
receive a second set of feature points detected based on a second frame;
generate a second image that indicates respective bounded neighborhoods each of which corresponds to a neighborhood surrounding one of the second set of feature points, wherein the respective bounded neighborhoods are indicated in the second image by the second binary value and a remainder of the second image is indicated by the first binary value; and
compare the first image to the second image to generate a third image, wherein locations of a subset of the first set of feature points excluded from the respective bounded neighborhoods are indicated in the third image by a binary value different from a remainder of the third image.

9. The computer vision system of claim 8 , wherein the feature point identification circuit is further configured to generate a third set of feature points that includes the subset of the first set of feature points and the second set of feature points.

10. The computer vision system of claim 9 further comprising a sparse optical flow circuit coupled to the feature point identification circuit to receive the third set of feature points.

11. The computer vision system of claim 8 , wherein the feature point identification circuit is further configured to determine coordinates of the subset of the first set of feature points.

12. The computer vision system of claim 8 , wherein each of the bounded neighborhoods has a 3×3 pixel area or a 5×5 pixel area.

13. The computer vision system of claim 8 , wherein the first binary value is a binary one and the second binary value is a binary zero.

14. The computer vision system of claim 8 ,
wherein dimensions of the first image are equal to dimensions of the first frame, and
wherein dimensions of the second image are equal to the dimensions of the first frame.

15. A non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to:
detect a first set of feature points within a first frame;
generate a first image that indicates locations of the first set of feature points detected within the first frame, wherein the locations of the first set of feature points are indicated in the first image by a first binary value and a remainder of the first image is indicated by a second binary value;
generate a second image that indicates respective bounded neighborhoods each of which corresponds to a neighborhood surrounding one of a second set of feature points detected based on a second frame, wherein the respective bounded neighborhoods are indicated in the second image by the second binary value and a remainder of the second image is indicated by the first binary value; and
compare the first image to the second image to generate a third image, wherein locations of a subset of the first set of feature points excluded from the respective bounded neighborhoods are indicated in the third image by a binary value different from a remainder of the third image.

16. The non-transitory computer readable medium of claim 15 comprising further instructions that cause the one or more processors to determine a set of coordinates for each feature point of the subset.

17. The non-transitory computer readable medium of claim 15 , comprising further instructions that cause the one or more processors to generate a third set of feature points that includes the subset to the first set of feature points and the second set of feature points.

18. The non-transitory computer readable medium of claim 15 , wherein the second frame is captured by a camera prior to the first frame.

19. The non-transitory computer readable medium of claim 15 , wherein each of the bounded neighborhoods has an a 3×3 pixel area or a 5×5 pixel area.

20. The non-transitory computer readable medium of claim 15 , wherein the first binary value is a binary one and the second binary value is a binary zero.

Assignees

• Texas Instruments Inc

Inventors

• Poddar Deepak Kumar
• Jain Anshu
• Kumar Desappan
• Swami Pramod Kumar

Classifications

• G06T7/246 Primary

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T2200/28

  involving image processing hardware · CPC title

• G06T2207/20016

  Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform · CPC title

• G06T2207/30241

  Trajectory · CPC title