Object detection and property determination for autonomous vehicles

Patent metadata

Field	Value
Publication number	US-12205030-B2
Application number	US-202318497025-A
Country	US
Kind code	B2
Filing date	Oct 30, 2023
Priority date	May 17, 2018
Publication date	Jan 21, 2025
Grant date	Jan 21, 2025

Abstract

Systems, methods, tangible non-transitory computer-readable media, and devices for detecting objects are provided. For example, the disclosed technology can obtain a representation of sensor data associated with an environment surrounding a vehicle. Further, the sensor data can include sensor data points. A point classification and point property estimation can be determined for each of the sensor data points and a portion of the sensor data points can be clustered into an object instance based on the point classification and point property estimation for each of the sensor data points. A collection of point classifications and point property estimations can be determined for the portion of the sensor data points clustered into the object instance. Furthermore, object instance property estimations for the object instance can be determined based on the collection of point classifications and point property estimations for the portion of the sensor data points clustered into the object instance.

First claim

What is claimed is:

1. A computer-implemented method comprising:
obtaining training data comprising a first portion corresponding to a representation of sensor data comprising a plurality of sensor data points, and a second portion indicative of ground truth data, wherein the ground truth data is indicative of a labeled object instance property estimation that corresponds with an object instance associated with the first portion;
inputting, into a machine-learned detector model, the first portion of the training data,
wherein the machine-learned detector model is configured to determine point property estimations for at least a portion of the plurality of sensor data points represented in the first portion of the training data and, based on the point property estimations, determine a predicted object instance property estimation for the object instance;
receiving, as an output of the machine-learned detector model, a training output indicative of the predicted object instance property estimation;
determining a loss based on the training output, wherein the loss is based on a comparison of the predicted object instance property estimation to the labeled object instance property estimation of the ground truth data; and
modifying the machine-learned detector model based on the loss.

2. The computer-implemented method of claim 1 , wherein the machine-learned detector model is configured to generate the object instance based on the point property estimations.

3. The computer-implemented method of claim 1 , wherein modifying the machine-learned detector model based on the loss comprises modifying one or more weights associated with the machine-learned detector model.

4. The computer-implemented method of claim 1 , wherein modifying the machine-learned detector model comprises performing a backpropagation for the machine-learned detector model based on the loss.

5. The computer-implemented method of claim 1 , wherein the object instance associated with the first portion comprises an object instance for an object that was identified based on the sensor data.

6. The computer-implemented method of claim 1 , wherein the labeled object instance property estimation comprise at least a bounding shape corresponding to the object instance associated with the first portion.

7. The computer-implemented method of claim 6 , wherein the labeled object instance property estimations are at least (i) manually annotated or (ii) automatically annotated.

8. An autonomous vehicle control system comprising:
one or more processors; and
one or more tangible, non-transitory computer readable media that store instructions that are executable by the one or more processors to cause the autonomous vehicle control system to perform operations, the operations comprising:
obtaining training data comprising a first portion corresponding to a representation of sensor data comprising a plurality of sensor data points, and a second portion indicative of ground truth data, wherein the ground truth data is indicative of a labeled object instance property estimation that corresponds with an object instance associated with the first portion;
inputting, into a machine-learned detector model, the first portion of the training data,
wherein the machine-learned detector model is configured to determine point property estimations for at least a portion of the plurality of sensor data points represented in the first portion of the training data and, based on the point property estimations, determine a predicted object instance property estimation for the object instance;
receiving, as an output of the machine-learned detector model, a training output indicative of the predicted object instance property estimation;
determining a loss based on the training output, wherein the loss is based on a comparison of the predicted object instance property estimation to the labeled object instance property estimation of the ground truth data; and
modifying the machine-learned detector model based on the loss.

9. The autonomous vehicle control system of claim 8 , wherein the machine-learned detector model is configured to generate the object instance based on the point property estimations.

10. The autonomous vehicle control system of claim 8 , wherein modifying the machine-learned detector model based on the loss comprises modifying one or more weights associated with the machine-learned detector model.

11. The autonomous vehicle control system of claim 8 , wherein modifying the machine-learned detector model comprises performing a backpropagation for the machine-learned detector model based on the loss.

12. The autonomous vehicle control system of claim 8 , wherein the object instance associated with the first portion comprises an object instance for an object that was identified based on the sensor data.

13. The autonomous vehicle control system of claim 8 , wherein the labeled object instance property estimation comprise at least a bounding shape corresponding to the object instance associated with the first portion.

14. The autonomous vehicle control system of claim 13 , wherein the labeled object instance property estimations are at least (i) manually annotated or (ii) automatically annotated.

15. One or more tangible, non-transitory computer readable media that store instructions that are executable by one or more processors to cause the one or more processors to perform operations, the operations comprising:
obtaining training data comprising a first portion corresponding to a representation of sensor data comprising a plurality of sensor data points, and a second portion indicative of ground truth data, wherein the ground truth data is indicative of a labeled object instance property estimation that corresponds with an object instance associated with the first portion;
inputting, into a machine-learned detector model, the first portion of the training data,
wherein the machine-learned detector model is configured to determine point property estimations for at least a portion of the plurality of sensor data points represented in the first portion of the training data and, based on the point property estimations, determine a predicted object instance property estimation for the object instance;
receiving, as an output of the machine-learned detector model, a training output indicative of the predicted object instance property estimation;
determining a loss based on the training output, wherein the loss is based on a comparison of the predicted object instance property estimation to the labeled object instance property estimation of the ground truth data; and
modifying the machine-learned detector model based on the loss.

16. The one or more tangible, non-transitory computer readable media of claim 15 , wherein the machine-learned detector model is configured to generate the object instance based on the point property estimations.

17. The one or more tangible, non-transitory computer readable media of claim 15 , wherein modifying the machine-learned detector model based on the loss comprises modifying one or more weights associated with the machine-learned detector model.

18. The one or more tangible, non-transitory computer readable media of claim 15 , wherein modifying the machine-learned detector model comprises performing a backpropagation for the machine-learned detector model based on the loss.

19. The one or more tangible, non-transitory computer readable media of claim 15 , wherein the object instance associated with the first portion comprises an object instance for an obj

Assignees

• Aurora Operations Inc

Inventors

• Kee Eric Randall
• Vallespi-Gonzalez Carlos
• Meyer Gregory P
• Laddha Ankit

Classifications

• G06V20/58 Primary

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G05D1/227

  Handing over between remote control and on-board control; Handing over between remote control arrangements · CPC title

• G06V10/776

  Validation; Performance evaluation · CPC title