Generating ground truth for machine learning from time series elements

Patent metadata

Field	Value
Publication number	US-11748620-B2
Application number	US-202117301965-A
Country	US
Kind code	B2
Filing date	Apr 20, 2021
Priority date	Feb 1, 2019
Publication date	Sep 5, 2023
Grant date	Sep 5, 2023

Abstract

Sensor data, including a group of time series elements, is received. A training data set is determined, including by determining for at least a selected time series element in the group of time series elements a corresponding ground truth. The corresponding ground truth is based on a plurality of time series elements in the group of time series elements. A processor is used to train a machine learning model using the training dataset.

First claim

What is claimed is:

1. A method implemented by one or more processors, wherein the method comprises:
obtaining sensor data captured at respective times within a period of time;
determining a ground truth based on the sensor data, the ground truth comprising a three-dimensional feature associated with the sensor data; and
training a machine learning model using a training dataset comprising the determined ground truth and a portion of the sensor data captured at a particular time within the period of time, wherein the machine learning model is trained to output the ground truth based on an input of the portion of the sensor data.

2. The method of claim 1 , wherein the sensor data comprises a group of time series elements associated with respective times within the period of time.

3. The method of claim 1 , wherein the three-dimensional feature is formed from portions of the sensor data captured at respective times.

4. The method of claim 1 , wherein the three-dimensional feature reflects a lane line.

5. The method of claim 4 , wherein the sensor data comprises a plurality of images captured at respective times, and wherein the lane line is formed from different portions of the lane line as depicted in the plurality of images.

6. The method of claim 5 , wherein a portion of an individual image depicting a portion of the lane line is selected based on a measure associated with relevancy of the portion of the individual image with respect to remaining images depicting the portion of the lane line.

7. The method of claim 4 , wherein the three dimensional feature reflects a three-dimensional trajectory of the lane line.

8. The method of claim 1 , wherein the three-dimensional feature reflects a path associated with a vehicle.

9. The method of claim 8 , wherein the sensor data comprises a plurality of images captured at respective times, and wherein the machine learning model is trained to output the path based on an individual image of the vehicle.

10. The method of claim 8 , wherein the vehicle is an adjacent lane to a different vehicle which captured the sensor data.

11. The method of claim 1 , wherein the ground truth is determined based on odometry information associated with the sensor data.

12. The method of claim 1 , wherein the training dataset further comprises scene data describing a real-world environment around a vehicle which captured the sensor data.

13. The method of claim 1 , wherein the portion of the sensor data is selected based on it being within a threshold number of remaining portions of the sensor data as ordered according to respective time of capture.

14. A system, comprising:
a processor; and
a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to:
obtain sensor data captured at respective times within a period of time;
determine a ground truth based on the sensor data, the ground truth comprising a three-dimensional feature associated with the sensor data; and
train a machine learning model using a training dataset comprising the determined ground truth and a portion of the sensor data captured at a particular time within the period of time, wherein the machine learning model is trained to output the ground truth based on an input of the portion of the sensor data.

15. The system of claim 14 , wherein the sensor data comprises a group of time series elements associated with respective times within the period of time.

16. The system of claim 14 , wherein the three-dimensional feature reflects a lane line.

17. The system of claim 16 , wherein the sensor data comprises a plurality of images captured at respective times, and wherein the lane line is formed from different portions of the lane line as depicted in the plurality of images.

18. The system of claim 17 , wherein a portion of an individual image depicting a portion of the lane line is selected based on a measure associated with relevancy of the portion of the individual image with respect to remaining images depicting the portion of the lane line.

19. The system of claim 14 , wherein the portion of the sensor data is selected based on it being within a threshold of number remaining portions of the sensor data as ordered according to respective time of capture.

20. A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions which when executed by a processor, cause the processor to:
obtain sensor data captured at respective times within a period of time;
determine a ground truth based on the sensor data, the ground truth comprising a three-dimensional feature associated with the sensor data; and
train a machine learning model using a training dataset comprising the determined ground truth and a portion of the sensor data captured at a particular time within the period of time, wherein the machine learning model is trained to output the ground truth based on an input of the portion of the sensor data.

Assignees

• Tesla Inc

Inventors

• Elluswamy Ashok Kumar
• Bauch Matthew
• Payne Christopher
• Karpathy Andrej
• Polin Joseph

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G05D2111/54

  for measuring the travel distances, e.g. by counting the revolutions of wheels · CPC title

• G05D2101/10

  using artificial intelligence [AI] techniques · CPC title

• G06F18/28

  Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries · CPC title