Generating ground truth for machine learning from time series elements

Patent metadata

Field	Value
Publication number	US-12223428-B2
Application number	US-202318459954-A
Country	US
Kind code	B2
Filing date	Sep 1, 2023
Priority date	Feb 1, 2019
Publication date	Feb 11, 2025
Grant date	Feb 11, 2025

Abstract

Sensor data, including a group of time series elements, is received. A training data set is determined, including by determining for at least a selected time series element in the group of time series elements a corresponding ground truth. The corresponding ground truth is based on a plurality of time series elements in the group of time series elements. A processor is used to train a machine learning model using the training dataset.

First claim

What is claimed is:

1. A method, comprising: receiving, by one or more processors, sensor data of a group of time series elements; determining, by the one or more processors, a training dataset, including by determining for at least a selected time series element in the group of time series elements a corresponding ground truth based on a plurality of time series elements in the group of time series elements; and training, by the one or more processors, using the training dataset, a machine learning model to output a predicted ground truth based on a single time series element, the training dataset comprising the selected time series element in the group of time series elements and the corresponding ground truth.

2. The method of claim 1 , wherein determining the training dataset further comprises identifying, by the one or more processors, a relevant portion of the selected time series element of the group of time series elements, wherein the relevant portion includes one or more of a lane line, a traffic control signal, and an object.

3. The method of claim 1 , wherein the corresponding ground truth is a three-dimensional representation of an attribute of the selected time series element of the group of time series elements.

4. The method of claim 3 , wherein the three-dimensional representation represents a vehicle lane, a predicted path of a vehicle, an obstacle, a traffic control signal, a map feature, an object distance, or a drivable space.

5. The method of claim 1 , wherein the sensor data of the group of time series elements comprises a plurality of images captured at respective times.

6. The method of claim 1 , further comprising determining, by the one or more processors, the corresponding ground truth based on the sensor data.

7. The method of claim 6 , wherein determining the corresponding ground truth comprises:
identifying, by the one or more processors, for at least one time series element, a respective portion of the at least one time series element to form the corresponding ground truth, and
generating, by the one or more processors, the corresponding ground truth based on the identified respective portion.

8. The method of claim 1 , wherein determining the training dataset further comprises:
selecting, by the one or more processors, an element of the group of time series elements;
associating, by the one or more processors, the element of the group of time series elements with the corresponding ground truth; and
including, by the one or more processors, the element and the associated corresponding ground truth in the training dataset.

9. The method of claim 1 , further comprising inferring, by the one or more processors, a feature of the sensor data based on the training dataset, wherein the feature includes at least one of a vehicle lane, a drivable space, a pedestrian, a stationary vehicle, a moving vehicle, rain, hail, fog, a traffic light, a traffic sign, a street sign, and a traffic pattern.

10. A system comprising: one or more processors; and a memory coupled with the one or more processors, wherein the memory is configured to provide the one or more processors with instructions which when executed cause the one or more processors to: receive sensor data of a group of time series elements; determine a training dataset, including by determining for at least a selected time series element in the group of time series elements a corresponding ground truth based on a plurality of time series elements in the group of time series elements; and train, using the training dataset, a machine learning model to output a predicted ground truth based on a single time series element, the training dataset comprising the selected time series element in the group of time series elements and the corresponding ground truth.

11. The system of claim 10 , wherein determining the training dataset further comprises identifying, by the one or more processors, a relevant portion of the selected time series element of the group of time series elements, wherein the relevant portion includes one or more of a lane line, a traffic control signal, and an object.

12. The system of claim 10 , wherein the corresponding ground truth is a three-dimensional representation of an attribute of the selected time series element of the group of time series elements.

13. The system of claim 12 , wherein the three-dimensional representation represents a vehicle lane, a predicted path of a vehicle, an obstacle, a traffic control signal, a map feature, an object distance, or a drivable space.

14. The system of claim 10 , wherein the sensor data of the group of time series elements comprises a plurality of images captured at respective times.

15. The system of claim 10 , further comprising determining, by the one or more processors, the corresponding ground truth based on the sensor data.

16. The system of claim 15 , wherein determining the corresponding ground truth comprises:
identifying, by the one or more processors, for at least one time series element, a respective portion of the at least one time series element to form the corresponding ground truth, and
generating, by the one or more processors, the corresponding ground truth based on the identified respective portion.

17. The system of claim 10 , wherein determining the training dataset further comprises:
selecting, by the one or more processors, an element of the group of time series elements;
associating, by the one or more processors, the element of the group of time series elements with the corresponding ground truth; and
including, by the one or more processors, the element and the associated corresponding ground truth in the training dataset.

18. The system of claim 10 , further comprising inferring, by the one or more processors, a feature of the sensor data based on the training dataset, wherein the feature includes at least one of a vehicle lane, a drivable space, a pedestrian, a stationary vehicle, a moving vehicle, rain, hail, fog, a traffic light, a traffic sign, a street sign, and a traffic pattern.

19. A non-transitory computer-readable storage medium comprising instructions which when executed by one or more processors, cause the one or more processors to: receive sensor data of a group of time series elements; determine a training dataset, including by determining for at least a selected time series element in the group of time series elements a corresponding ground truth based on a plurality of time series elements in the group of time series elements; and train, using the training dataset, a machine learning model to output a predicted ground truth based on a single time series element, the training dataset comprising the selected time series element in the group of time series elements and the corresponding ground truth.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the instructions further cause the one or more processors to:
select an element of the group of time series elements;
associate the element of the group of time series elements with the corresponding ground truth; and
include the element and the associated corresponding ground truth in the training dataset.

Assignees

• Tesla Inc

Inventors

• Elluswamy Ashok Kumar
• Bauch Matthew
• Payne Christopher
• Karpathy Andrej
• Polin Joseph

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G16Y20/10 Primary

  relating to the environment, e.g. temperature; relating to location · CPC title

• G06F18/28

  Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries · CPC title

• G06V20/588

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title