System and method for identifying travel way features for autonomous vehicle motion control

Patent metadata

Field	Value
Publication number	US-12248075-B2
Application number	US-202418672986-A
Country	US
Kind code	B2
Filing date	May 23, 2024
Priority date	Nov 16, 2018
Publication date	Mar 11, 2025
Grant date	Mar 11, 2025

Abstract

Systems and methods for identifying travel way features in real time are provided. A method can include receiving two-dimensional and three-dimensional data associated with the surrounding environment of a vehicle. The method can include providing the two-dimensional data as one or more input into a machine-learned segmentation model to output a two-dimensional segmentation. The method can include fusing the two-dimensional segmentation with the three-dimensional data to generate a three-dimensional segmentation. The method can include storing the three-dimensional segmentation in a classification database with data indicative of one or more previously generated three-dimensional segmentations. The method can include providing one or more datapoint sets from the classification database as one or more inputs into a machine-learned enhancing model to obtain an enhanced three-dimensional segmentation. And, the method can include identifying one or more travel way features based at least in part on the enhanced three-dimensional segmentation.

First claim

What is claimed is:

1. A computer-implemented method comprising:
obtaining three-dimensional data associated with an environment of an autonomous vehicle at a current time;
generating a current three-dimensional segmentation based on the three dimensional data associated with the environment of the autonomous vehicle at the current time;
obtaining a previously generated three-dimensional segmentation associated with the environment of the autonomous vehicle;
generating an updated three-dimensional segmentation based on the current three-dimensional segmentation and the previously generated three-dimensional segmentation;
determining a travel way feature within the environment based on the updated three-dimensional segmentation; and
controlling an operation of the autonomous vehicle based on the travel way feature within the environment.

2. The computer-implemented method of claim 1 , comprising:
providing, for storage in a memory, the current three-dimensional segmentation, wherein storage of the current three-dimensional segmentation in the memory updates the previously generated three-dimensional segmentation.

3. The computer-implemented method of claim 1 , wherein the updated three-dimensional segmentation comprises a plurality of three-dimensional datapoints, the plurality of three-dimensional datapoints indicative of an estimated segmentation of aggregated three-dimensional points at the current time.

4. The computer-implemented method of claim 3 , wherein the plurality of datapoints correspond to the travel way feature associated with the previously generated three-dimensional segmentation.

5. The computer-implemented method of claim 1 , comprising:
iteratively updating the current three-dimensional segmentation at one or more times succeeding the current time.

6. The computer-implemented method of claim 1 , wherein the operation of the autonomous vehicle comprises controlling a motion of the autonomous vehicle, the method comprising:
initiating a motion control of the autonomous vehicle based on the travel way feature within the environment.

7. The computer-implemented method of claim 6 , comprising:
generating a motion plan based on the travel way feature within the environment; and
providing, to a vehicle control system, data based on the motion plan of the autonomous vehicle.

8. The computer-implemented method of claim 1 , wherein the previously generated three-dimensional segmentation and the updated three-dimensional segmentation are stored in non-parametric memory.

9. The computer-implemented method of claim 1 , wherein the previously generated three-dimensional segmentation and the updated three-dimensional segmentation are stored remote from the autonomous vehicle.

10. An autonomous vehicle control system comprising:
one or more processors; and
one or more tangible, non-transitory, computer readable media storing instructions that are executable by the one or more processors to cause the autonomous vehicle control system to perform operations, the operations comprising:
obtaining three-dimensional data associated with an environment of an autonomous vehicle at a current time;
generating a current three-dimensional segmentation based on the three dimensional data associated with the environment of the autonomous vehicle at the current time;
obtaining a previously generated three-dimensional segmentation associated with the environment of the autonomous vehicle;
generating an updated three-dimensional segmentation based on the current three-dimensional segmentation and the previously generated three-dimensional segmentation;
determining a travel way feature within the environment based on the updated three-dimensional segmentation; and
controlling the autonomous vehicle based on the travel way feature within the environment.

11. The autonomous vehicle control system of claim 10 , wherein the operations comprise:
providing, for storage in a memory, the current three-dimensional segmentation, wherein storage of the current three-dimensional segmentation in the memory updates the previously generated three-dimensional segmentation.

12. The autonomous vehicle control system of claim 10 , wherein the operations comprise:
iteratively updating the current three-dimensional segmentation at one or more times succeeding the current time.

13. The autonomous vehicle control system of claim 10 , wherein the operations comprise:
initiating a motion control of the autonomous vehicle based on the travel way feature within the environment.

14. The autonomous vehicle control system of claim 10 , wherein the previously generated three-dimensional segmentation and the updated three-dimensional segmentation are stored in non-parametric memory.

15. The autonomous vehicle control system of claim 10 , wherein the previously generated three-dimensional segmentation and the updated three-dimensional segmentation are stored remote from the autonomous vehicle.

16. One or more tangible, non-transitory, computer readable media storing instructions that are executable by one or more processors to perform operations, the operations comprising:
obtaining three-dimensional data associated with an environment of an autonomous vehicle at a current time;
generating a current three-dimensional segmentation based on the three dimensional data associated with the environment of the autonomous vehicle at the current time;
obtaining a previously generated three-dimensional segmentation associated with the environment of the autonomous vehicle;
generating an updated three-dimensional segmentation based on the current three-dimensional segmentation and the previously generated three-dimensional segmentation;
determining a travel way feature within the environment based on the updated three-dimensional segmentation; and
controlling the autonomous vehicle based on the travel way feature within the environment.

17. The one or more tangible, non-transitory, computer readable media of claim 16 , wherein the operations comprise:
providing, for storage in a memory, the current three-dimensional segmentation, wherein storage of the current three-dimensional segmentation in the memory updates the previously generated three-dimensional segmentation.

18. The one or more tangible, non-transitory, computer readable media of claim 17 , wherein the memory stores one or more past observations associated with the environment of the autonomous vehicle.

19. The one or more tangible, non-transitory, computer readable media of claim 16 , wherein the operations comprise:
initiating a motion control of the autonomous vehicle based on the travel way feature within the environment; and
initiating a motion of the autonomous vehicle based on the travel way feature within the environment.

20. The one or more tangible, non-transitory, computer readable media of claim 16 , wherein the operations comprise:
iteratively updating the current three-dimensional segmentation at one or more times succeeding the current time.

Assignees

• Aurora Operations Inc

Inventors

• Urtasun Raquel
• Bai Min
• Wang Shenlong

Classifications

• G06V20/582

  of traffic signs · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06V10/26

  Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion · CPC title

• G06V10/803

  of input or preprocessed data · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title