Identifying and/or removing false positive detections from LIDAR sensor output

Patent metadata

Field	Value
Publication number	US-11740335-B2
Application number	US-202016851060-A
Country	US
Kind code	B2
Filing date	Apr 16, 2020
Priority date	Mar 27, 2019
Publication date	Aug 29, 2023
Grant date	Aug 29, 2023

Abstract

A machine-learned (ML) model for detecting that depth data (e.g., lidar data, radar data) comprises a false positive attributable to particulate matter, such as dust, steam, smoke, rain, etc. The ML model may be trained based at least in part on simulated depth data generated by a fluid dynamics model and/or by collecting depth data during operation of a device (e.g., an autonomous vehicle. In some examples, an autonomous vehicle may identify depth data that may be associated with particulate matter based at least in part on an outlier region in a thermal image. For example, the outlier region may be associated with steam.

First claim

What is claimed is:

1. A non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving sensor data associated with an environment, the sensor data indicating one or more depth measurements;
providing at least a portion of the sensor data to a machine-learned (ML) model as input, the ML model being trained to classify a false positive or a true positive associated with particulate matter based on the portion of the sensor data;
receiving, from the ML model, an indication that a depth measurement of the one or more depth measurements of the portion is associated with the particulate matter and comprises the false positive associated with the particulate matter; and
controlling an autonomous vehicle based at least in part on the indication.

2. The non-transitory computer-readable medium of claim 1 , wherein:
receiving the sensor data comprises receiving thermal data, the thermal data indicating a temperature associated with a location in the environment; and
the operations further comprise determining, based at least in part on the thermal data, the portion of the sensor data.

3. The non-transitory computer-readable medium of claim 2 , wherein:
the ML model is a first ML model of a plurality of models trained to classify sensor data as either particulate matter or a first classification;
a second ML of the plurality of models is trained to classify sensor data as either particulate matter or a second classification; and
the operations further comprise inputting the portion into the second ML model.

4. The non-transitory computer-readable medium of claim 2 , wherein the operations further comprise:
receiving, from the ML model, a first indication that a first sensor data point is a false positive; and
receiving, from the ML model, a second indication that a second sensor data point is a true positive,
wherein the first sensor data point and the second sensor data point are provided to the ML model as input.

5. The non-transitory computer-readable medium of claim 2 , wherein the operations further comprise adding at least one of a portion of the thermal data, the portion of the sensor data, or the indication to a training data set for training the ML model.

6. The non-transitory computer-readable medium of claim 5 , the operations further comprising:
transmitting the training data set; and
receiving an updated ML model based at least in part on the training data set.

7. The non-transitory computer-readable medium of claim 1 , wherein the operations further comprising receiving, from the ML model, a confidence associated with the indication.

8. A method comprising:
receiving sensor data associated with an environment, the sensor data indicating one or more depth measurements;
providing at least a portion of the sensor data to a machine-learned (ML) model as input, the ML model being trained to classify a false positive or a true positive associated with particulate matter based on the portion of the sensor data;
receiving, from the ML model, an indication that a depth measurement of the one or more depth measurements of the portion is associated with particulate matter and comprises the false positive associated with the particulate matter; and
controlling an autonomous vehicle based at least in part on the indication.

9. The method of claim 8 , wherein:
receiving the sensor data comprises receiving thermal data, the thermal data indicating a temperature associated with a location in the environment; and
the method further comprising determining, based at least in part on the thermal data, the portion of the sensor data.

10. The method of claim 9 , wherein:
the ML model is a first ML model of a plurality of models trained to classify sensor data as either particulate matter or a first classification;
a second ML model of the plurality of models is trained to classify sensor data as either particulate matter or a second classification; and
the method further comprises inputting the portion into the second ML model.

11. The method of claim 9 , wherein the method further comprises:
receiving, from the ML model, a first indication that a first sensor data point is a false positive; and
receiving, from the ML model, a second indication that a second sensor data point is a true positive,
wherein the first sensor data point and the second sensor data point are provided to the ML model as input.

12. The method of claim 9 , wherein the method further comprises adding at least one of a portion of the thermal data, the portion of the sensor data, or the indication to a training data set for training the ML model.

13. The method of claim 12 , the methods further comprise:
transmitting the training data set; and
receiving an updated ML model based at least in part on the training data set.

14. The method of claim 8 further comprising receiving, from the ML model, a confidence associated with the indication.

15. A system comprising:
one or more processors; and
a non-transitory memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
receiving first sensor data associated with an environment, the first sensor data indicating one or more depth measurements;
providing at least a portion of the first sensor data to a machine-learned (ML) model as input, the ML model being trained to classify a false positive or a true positive associated with particulate matter based on the portion of the sensor data;
receiving, from the ML model, an indication that a depth measurement of the one or more depth measurements of the portion is associated with the particulate matter and comprises the false positive associated with particulate matter; and
controlling an autonomous vehicle based at least in part on the indication.

16. The system of claim 15 , wherein the operations further comprise:
receiving second sensor data comprising thermal data, the thermal data indicating a temperature associated with a location in the environment; and
the operations further comprise determining, based at least in part on the thermal data, the portion of the first sensor data.

17. The system of claim 16 , wherein providing at least the portion of the sensor data to the ML model as input is based at least in part on determining, based at least in part on the thermal data, the portion of the sensor data.

18. The system of claim 15 , wherein:
the ML model is a first ML model of a plurality of models trained to classify sensor data as either particulate matter or a first classification;
a second ML model of the plurality of models is trained to classify sensor data as either particulate matter or a second classification; and
the operations further comprise inputting the portion into the second ML model.

19. The system of claim 15 , wherein the operations further comprise:
receiving, from the ML model, a first indication that a first sensor data point is a false positive; and
receiving, from the ML model, a second indication that a second sensor data point is a true positive,
wherein the first sensor data point and the second sensor data point are provided to the ML model as input.

20. The system of claim 15 , wherein the operations further comprising receiving, from the ML model, a confidence associated with the indication.

Assignees

• Zoox Inc

Inventors

• Bhaskaran Sreevatsan
• Ferdowsi Mehran
• Mcmichael Ryan
• Subasingha Subasingha Shaminda

Classifications

• G05D1/0257

  using a radar (radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931) · CPC title

• G05D1/0231

  using optical position detecting means (position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16) · CPC title

• G05D1/0088

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title