Advanced path prediction

Patent metadata

Field	Value
Publication number	US-10871783-B2
Application number	US-201816138059-A
Country	US
Kind code	B2
Filing date	Sep 21, 2018
Priority date	Mar 23, 2016
Publication date	Dec 22, 2020
Grant date	Dec 22, 2020

Abstract

The present disclosure provides systems and methods for mapping a determined path of travel. The path of travel may be mapped to a camera view of a camera affixed to a vehicle. In some embodiments, the path of travel may be mapped to another view that is based on a camera, such as a bird's eye view anchored to the camera's position at a given time. These systems and methods may determine the path of travel by incorporating data from later points in time.

First claim

What is claimed is:

1. A method comprising:
determining an actual path of travel of a vehicle from a first time to a second time;
receiving a captured image, wherein the captured image is captured by a camera at the first time, and wherein the camera is mounted to the vehicle;
generating a first reference frame associated with the first time from the captured image using an inverse perspective model, wherein the first reference frame is an image derived from the captured image, and wherein the first reference frame is a bird's eye view reference frame anchored to a position of the camera; and
mapping, at or after the second time, the actual path of travel to the first reference frame.

2. The method of claim 1 , wherein mapping the actual path comprises:
orienting the actual path to align with the bird's eye view reference frame; and
scaling the actual path based on a scale of the bird's eye view reference frame.

3. The method of claim 2 , further comprising:
determining a perspective mapping from the bird's eye view reference frame to a camera reference frame; and
mapping the actual path from the first reference frame to the camera reference frame based on the perspective mapping.

4. The method of claim 1 , further comprising:
receiving visual data from the camera at the first time;
determining a location of an object based on the visual data; and
determining a distance between the object and the actual path of travel in the first reference frame.

5. The method of claim 4 , wherein the object is at least one of a second vehicle, a lane boundary, a pedestrian, a traffic sign, an animal, or a road debris.

6. The method of claim 4 , further comprising:
determining a driver behavior metric based at least in part on the determined distance.

7. The method of claim 6 , wherein the object is a second vehicle, and wherein the driver behavior metric is a tailgating score based on the determined distance.

8. The method of claim 1 , further comprising:
predicting a second path of travel of the vehicle;
determining a difference between the predicted second path and the actual path.

9. The method of claim 8 , further comprising:
determining a driver behavior metric based on the determined difference.

10. The method of claim 8 , further comprising:
training a path prediction model based on the determined difference.

11. The method of claim 1 , further comprising:
receiving visual data from the camera at the first time;
estimating a lane location at the first time based on the visual data; and
refining the estimated lane location based on the actual path of travel to produce a refined lane location.

12. The method of claim 11 , further comprising:
training a lane detection model based on the estimated lane location and the refined lane location.

13. The method of claim 1 , further comprising:
receiving visual data from the camera at the first time;
determining a plurality of regions of interest in the visual data based on the actual path of travel;
processing a first region of interest of the plurality of regions of interest at a first resolution;
processing a second region of interest of the plurality of regions of interest at a second resolution; and
detecting at least one object in at least one the first region of interest or the second region of interest.

14. The method of claim 13 , wherein the first resolution is based on an estimated distance of objects within the first region of interest.

15. The method of claim 14 , wherein the estimated distance of the objects is based on at least one of a typical lane width in the first region of interest, or the actual path of travel to the first region of interest, wherein the typical lane width is 3.7 meters.

16. An apparatus comprising:
at least one memory unit; and
at least one processor coupled to the at least one memory unit, in which the at least one processor is configured to:
determine an actual path of travel of a vehicle from a first time to a second time;
receive a captured image, wherein the captured image is captured by a camera at the first time, and wherein the camera is mounted to the vehicle;
generate a first reference frame associated with the first time from the captured image using an inverse perspective model, wherein the first reference frame is an image derived from the captured image, and wherein the first reference frame is a bird's eye view reference frame anchored to a position of the camera; and
map, at or after the second time, the actual path of travel to the first reference frame.

17. A computer program product, the computer program product comprising:
a non-transitory computer-readable medium having program code recorded thereon, the program code, when executed by a processor, causes the processor to:
determine an actual path of travel of a vehicle from a first time to a second time;
receive a captured image, wherein the captured image is captured by a camera at the first time, and wherein the camera is mounted to the vehicle;
generate a first reference frame associated with the first time from the captured image using an inverse perspective model, wherein the first reference frame is an image derived from the captured image, and wherein the first reference frame is a bird's eye view reference frame anchored to a position of the camera; and
map, at or after the second time, the actual path of travel to the first reference frame.

Assignees

• Netradyne Inc

Inventors

• Julian David Jonathan
• Agrawal Avneesh
• Annapureddy Venkata Sreekanta Reddy

Classifications

• G06V10/82 Primary

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2413

  based on distances to training or reference patterns · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title