Generating ground truth for machine learning from time series elements

Patent metadata

Field	Value
Publication number	US-10997461-B2
Application number	US-201916265729-A
Country	US
Kind code	B2
Filing date	Feb 1, 2019
Priority date	Feb 1, 2019
Publication date	May 4, 2021
Grant date	May 4, 2021

Abstract

Sensor data, including a group of time series elements, is received. A training data set is determined, including by determining for at least a selected time series element in the group of time series elements a corresponding ground truth. The corresponding ground truth is based on a plurality of time series elements in the group of time series elements. A processor is used to train a machine learning model using the training dataset.

First claim

What is claimed is:

1. A method, comprising:
receiving sensor data including a group of time series elements captured at respective times within a period of time;
determining a ground truth for the group of time series elements, wherein determining the ground truth comprises:
identifying, for individual time series elements, respective portions of the individual time series elements to form the ground truth, and
generating the ground truth based on the identified portions; and

using a processor to train a machine learning model using a training dataset comprising the determined ground truth and a selected time series element in the group of time series elements, wherein the machine learning model is trained to output the ground truth for the group of time series elements based on an input of the selected time series element.

2. The method of claim 1 , wherein the determined ground truth is associated with a vehicle lane line.

3. The method of claim 2 , wherein identifying respective portions of the individual time series elements comprises identifying different portions of the vehicle lane line from different elements of the group of time series elements.

4. The method of claim 1 , wherein the group of time series elements are used to identify locations of a vehicle lane line in the selected time series element.

5. The method of claim 1 , wherein each element of the group of time series elements includes an image associated with a corresponding timestamp.

6. The method of claim 1 , wherein the ground truth is determined using odometry data associated with the group of time series elements.

7. The method of claim 6 , wherein the odometry data includes a vehicle position data and a vehicle orientation data.

8. The method of claim 6 , wherein the odometry data identifies a first change in a vehicle position and a second change in a vehicle orientation.

9. The method of claim 1 , wherein the determined ground truth represents a three-dimensional trajectory of a lane line.

10. The method of claim 9 , wherein the three-dimensional trajectory is represented as a parametric curve.

11. The method of claim 1 , wherein the determined ground truth is associated with a predicted path of a second vehicle different from a first vehicle that includes sensors that captured the sensor data.

12. The method of claim 11 , wherein the second vehicle is identified as entering a lane of the first vehicle.

13. The method of claim 1 , wherein the determined ground truth is associated with a distance of an object.

14. The method of claim 13 , wherein the object is an obstacle, a moving vehicle, a stationary vehicle, or a barrier.

15. The method of claim 13 , wherein the distance of the object is determined based on radar data associated with the group of time series elements.

16. The method of claim 1 , wherein a number of the elements included in the group of time series elements is based on a distance traveled.

17. The method of claim 1 , wherein a number of the elements included in the group of time series elements is based on an average vehicle speed.

18. The method of claim 1 , wherein identifying a portion of an individual time series element is based on an accuracy measure associated with a feature depicted in the ground truth.

19. A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions which when executed by a processor, cause the processor to:
receive sensor data including a group of time series elements;
determine a ground truth for the group of time series elements, wherein to determine the ground truth the computer instructions cause the processor to:
identify, for individual time series elements, respective portions of the individual time series elements to form the ground truth, and
generate the ground truth based on the identified portions; and

form a training dataset, wherein the training data set is limited to the determined ground truth for the group of time series elements and a selected time series element, wherein the selected time series element is a single time series element of the group of time series elements,
and wherein the training dataset is configured to train a machine learning model to output the ground truth for the group of time series elements based on an input of the single time series element.

20. A system, comprising:
a processor; and
a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to:
receive sensor data including a group of time series elements;
determine a ground truth for the group of time series elements, wherein to determine the ground truth the instructions cause the processor to:
identify, for individual time series elements, respective portions of the individual time series elements to form the ground truth, and
generate the ground truth based on the identified portions; and

train a machine learning model using a training dataset comprising the determined ground truth and a selected time series element in the group of time series elements, wherein the machine learning model is trained to output the ground truth for the group of time series elements based on an input of the selected time series element.

Assignees

• Tesla Inc

Inventors

• Elluswamy Ashok Kumar
• Bauch Matthew
• Payne Christopher
• Karpathy Andrej
• Polin Joseph

Classifications

• G06F18/28

  Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G05D2111/54

  for measuring the travel distances, e.g. by counting the revolutions of wheels · CPC title

• G05D2101/10

  using artificial intelligence [AI] techniques · CPC title