Estimation of time to collision in a computer vision system

Patent metadata

Field	Value
Publication number	US-11615629-B2
Application number	US-202117195915-A
Country	US
Kind code	B2
Filing date	Mar 9, 2021
Priority date	Oct 19, 2016
Publication date	Mar 28, 2023
Grant date	Mar 28, 2023

Abstract

A method for estimating time to collision (TTC) of a detected object in a computer vision system is provided that includes determining a three dimensional (3D) position of a camera in the computer vision system, determining a 3D position of the detected object based on a 2D position of the detected object in an image captured by the camera and an estimated ground plane corresponding to the image, computing a relative 3D position of the camera, a velocity of the relative 3D position, and an acceleration of the relative 3D position based on the 3D position of the camera and the 3D position of the detected object, wherein the relative 3D position of the camera is relative to the 3D position of the detected object, and computing the TTC of the detected object based on the relative 3D position, the velocity, and the acceleration.

First claim

What is claimed is:

1. A method comprising:
computing, based on an image captured by a camera on a vehicle, a distance of an object from the camera;
computing a time to collision of the object based on the distance;
determining an angle of approach of the object;
determining an angle threshold based on the distance;
comparing the angle of approach of the object to the angle threshold to determine that the object is in a trajectory of the camera; and
outputting the time to collision in response to determining that the object is in the trajectory of the vehicle.

2. The method of claim 1 ,
wherein the angle threshold is a percentage of a predetermined angle threshold, and
wherein the percentage is decreased as the distance increases.

3. The method of claim 2 , wherein comparing the angle of approach to the angle threshold comprises:
comparing the angle of approach to the predetermined angle threshold if the distance is less than a first predetermined distance;
comparing the angle of approach to a first percentage of the predetermined angle threshold if the distance is between the first predetermined distance and a second predetermined distance, wherein the second predetermined distance is larger than the first predetermined distance;
comparing the angle of approach to a second percentage of the predetermined angle threshold if the distance is between the second predetermined distance and a third predetermined distance, wherein the third predetermined distance is larger than the second predetermined distance and the second percentage is smaller than the first percentage; and
comparing the angle of approach to a third percentage of the predetermined angle threshold if the distance is greater than the third predetermined distance, wherein the third percentage is smaller than the second percentage.

4. The method of claim 3 , wherein the first percentage is 75%, the second percentage is 50%, and the third percentage is 25%.

5. The method of claim 1 , wherein computing the time to collision further comprises computing the time to collision based on a projection of a relative velocity of the camera in a direction of the object.

6. The method of claim 1 , further comprising computing a velocity of the camera relative to the object,
wherein computing the time to collision is based on the distance and the velocity of the camera relative to the object.

7. The method of claim 1 , further comprising determining a three dimensional position of the camera from a camera pose matrix corresponding to the image.

8. A computer system comprising a processor; and a memory coupled to the processor comprising instructions executable by the processor, the processor being operable when executing the instructions to:
capture an image by a camera on a vehicle;
compute a distance of an object from the camera based on the image;
compute a time to collision of the object based on the distance;
determine an angle of approach of the object;
determine an angle threshold based on the distance;
compare the angle of approach of the object to the angle threshold to determine that the object is in a trajectory of the camera; and
output the time to collision in response to determining that the object is in the trajectory of the vehicle.

9. The computer system of claim 8 , wherein the processor is further operable when executing the instructions to generate a three dimensional point cloud corresponding to the image.

10. The computer system of claim 9 , wherein the processor is further operable when executing the instructions to:
generate an estimated ground plane based on the three dimensional point cloud; and
determine a position of the object based on the estimated ground plane.

11. The computer system of claim 8 , wherein the processor is further operable when executing the instructions to compute the time to collision based on a projection of a relative velocity of the camera in a direction of the object.

12. The computer system of claim 8 , wherein the processor is further operable when executing the instructions to compute a velocity of the camera relative to the object,
wherein the processor is configured to compute the time to collision based on the distance and the velocity of the camera relative to the object.

13. The computer system of claim 8 ,
wherein the angle threshold is a percentage of a predetermined angle threshold, and
wherein the percentage is decreased as the distance increases.

14. The computer system of claim 13 , wherein the instructions to compare the angle of approach to the angle threshold comprise:
instructions to compare the angle of approach to the predetermined angle threshold if the distance is less than a first predetermined distance;
instructions to compare the angle of approach to a first percentage of the predetermined angle threshold if the distance is between the first predetermined distance and a second predetermined distance, wherein the second predetermined distance is larger than the first predetermined distance;
instructions to compare the angle of approach to a second percentage of the predetermined angle threshold if the distance is between the second predetermined distance and a third predetermined distance, wherein the third predetermined distance is larger than the second predetermined distance and the second percentage is smaller than the first percentage; and
instructions to compare the angle of approach to a third percentage of the predetermined angle threshold if the distance is greater than the third predetermined distance, wherein the third percentage is smaller than the second percentage.

15. The computer system of claim 14 , wherein the first percentage is 75%, the second percentage is 50%, and the third percentage is 25%.

16. A system-on-a-chip (SOC) comprising:
a general purpose processor (GPP);
a digital signal processor (DSP);
a vision processor; and
a memory configured to store instructions executable by the GPP, by the DSP, or by the vision processor,
wherein the GPP, the DSP, or the vision processor is configured to, when executing the instructions to:
capture an image by a camera on a vehicle;
compute a distance of an object from the camera based on the image;
compute a time to collision of the object based on the distance;
determine an angle of approach of the object;
determine an angle threshold based on the distance;
compare the angle of approach of the object to the angle threshold to determine that the object is in a trajectory of the camera; and
output the time to collision in response to determining that the object is in the trajectory of the vehicle.

17. The SOC of claim 16 , wherein the GPP, the DSP, or the vision processor is further configured, when executing the instructions, to compute the time to collision further comprises computing the time to collision based on a projection of a relative velocity of the camera in a direction of the object.

18. The SOC of claim 16 , wherein the GPP, the DSP, or the vision processor is further configured, when executing the instructions, to generate a three dimensional point cloud corresponding to the image.

19. The SOC of claim 18 , wherein the GPP, the DSP, or the vision processor is further configured to, when executing the instructions:
generate an estimated ground plane based on the three dimensional point cloud; and
determine a position of the object based on the estimated ground plane.

20. The SOC of claim 16 ,
wherein the angle threshold is a percentage of a predetermined

Assignees

• Texas Instruments Inc

Inventors

• Viswanath Prashanth Ramanathpur
• Poddar Deepak Kumar
• Nagori Soyeb
• Mathew Manu

Classifications

• G06T2207/30241

  Trajectory · CPC title

• G01C11/02 Primary

  Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T7/248

  involving reference images or patches · CPC title

• G01B11/00

  Measuring arrangements characterised by the use of optical techniques · CPC title