Estimation of time to collision in a computer vision system

Patent metadata

Field	Value
Publication number	US-10977502-B2
Application number	US-201916272415-A
Country	US
Kind code	B2
Filing date	Feb 11, 2019
Priority date	Oct 19, 2016
Publication date	Apr 13, 2021
Grant date	Apr 13, 2021

Abstract

A method for estimating time to collision (TTC) of a detected object in a computer vision system is provided that includes determining a three dimensional (3D) position of a camera in the computer vision system, determining a 3D position of the detected object based on a 2D position of the detected object in an image captured by the camera and an estimated ground plane corresponding to the image, computing a relative 3D position of the camera, a velocity of the relative 3D position, and an acceleration of the relative 3D position based on the 3D position of the camera and the 3D position of the detected object, wherein the relative 3D position of the camera is relative to the 3D position of the detected object, and computing the TTC of the detected object based on the relative 3D position, the velocity, and the acceleration.

First claim

What is claimed is:

1. A method comprising:
determining a three dimensional (3D) position of a camera in a computer system;
determining a 3D position of an object based on a 2D position of the object in an image captured by the camera and an estimated ground plane corresponding to the image;
computing a relative 3D position of the camera and a velocity of the relative 3D position based on the 3D position of the camera and the 3D position of the object, wherein the relative 3D position of the camera is relative to the 3D position of the object; and
computing a time to collision of the object based on the relative 3D position and the velocity of the relative 3D position.

2. The method of claim 1 , further comprising determining if the object is in a trajectory of a vehicle comprising the camera, the determining comprising comparing an angle of approach of the object to an angle threshold.

3. The method of claim 2 , wherein comparing an angle of approach comprises comparing the angle of approach to an angle threshold determined based on a distance of the object from the camera.

4. The method of claim 3 , wherein the angle threshold is a percentage of a predetermined angle threshold, wherein the percentage is decreased as the distance increases.

5. The method of claim 4 , wherein comparing an angle of approach comprises:
comparing the angle of approach to the predetermined angle threshold if the distance is less than a first predetermined distance;
comparing the angle of approach to a first percentage of the predetermined angle threshold if the distance is between the first predetermined distance and a second predetermined distance, wherein the second predetermined distance is larger than the first predetermined distance;
comparing the angle of approach to a second percentage of the predetermined angle threshold if the distance is between the second predetermined distance and a third predetermined distance, wherein the third predetermined distance is larger than the second predetermined distance and the second percentage is larger than the first percentage; and
comparing the angle of approach to a third percentage of the predetermined angle threshold if the distance is greater than the third predetermined distance, wherein the third percentage is larger than the second percentage.

6. The method of claim 5 , wherein the first percentage is 25%, the second percentage is 50%, and the third percentage is 75%.

7. The method of claim 1 , wherein the 2D position of the object is a midpoint between bottom two corners of a bounding box corresponding to the object.

8. The method of claim 1 , wherein the 3D position of the camera is determined from a camera pose matrix corresponding to the image.

9. The method of claim 1 , wherein computing the TTC further comprises computing the TTC based on a projection of the velocity of the relative 3D position of the camera in a direction of the object.

10. A computer system comprising a processor; and a memory coupled to the processor comprising instructions executable by the processor, the processor being operable when executing the instructions to:
capture a two dimensional (2D) image of a scene by a camera;
determine a three dimensional (3D) position of the camera corresponding to the 2D image;
determine a 3D position of san object based on a 2D position of the object in the 2D image and an estimated ground plane corresponding to the 2D image;
compute a relative 3D position of the camera and a velocity of the relative 3D position based on the 3D position of the camera and the 3D position of the object, wherein the relative 3D position of the camera is relative to the 3D position of the object; and
compute a time to collision of the object based on the relative 3D position and the velocity of the relative 3D position.

11. The computer system of claim 10 , wherein the processor is further operable when executing the instructions to determine if the object is in a trajectory of a vehicle comprising the camera, the determining comprising comparing an angle of approach of the object to an angle threshold.

12. The computer system of claim 11 , wherein the processor is further operable when executing the instructions to compare the angle of approach to an angle threshold determined based on a distance of the object from the camera.

13. The computer system of claim 10 , wherein the 2D position of the object is a midpoint between bottom two corners of a bounding box corresponding to the object.

14. The computer system of claim 10 , wherein the processor is further operable when executing the instructions to generate a 3D point cloud corresponding to the 2D image.

15. The computer system of claim 14 , wherein the processor is further operable when executing the instructions to generate the estimated ground plane based on the 3D point cloud.

16. The computer system of claim 10 , wherein the processor is further operable when executing the instructions to compute the TTC based on a projection of the velocity of the relative 3D position of the camera in a direction of the object.

17. A method comprising:
computing a distance of an object from a camera in the computer system, wherein the object is in a field of view of the camera;
computing a projected velocity of a relative three dimensional (3D) position of the camera in a direction of the object, wherein the relative 3D position of the camera is relative to a 3D position of the object;
and
computing a time to collision based on the distance of the object and the projected velocity of the relative 3D position.

18. The method of claim 17 , wherein computing the time to collision further comprises:
computing the time to collision as the distance divided by the projected velocity.

19. The method of claim 17 , further comprising:
determining a 3D position of the camera;
determining the 3D position of the object based on a 2D position of the object in a two dimensional (2D) image captured by the camera and an estimated ground plane corresponding to the 2D image; and
computing the relative 3D position of the camera and a velocity of the relative 3D position based on the 3D position of the camera and the 3D position of the object, wherein the relative 3D position of the camera is relative to the 3D position of the object.

20. The method of claim 19 , further comprising computing a projected acceleration of the object as a product of the relative 3D position of the camera and the acceleration of the relative 3D position divided by the distance.

Assignees

• Texas Instruments Inc

Inventors

• Viswanath Prashanth Ramanathpur
• Poddar Deepak Kumar
• Nagori Soyeb
• Mathew Manu

Classifications

• G06V20/58 Primary

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G01B11/00

  Measuring arrangements characterised by the use of optical techniques · CPC title

• G01C11/02 Primary

  Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06T7/248

  involving reference images or patches · CPC title