Systems and methods for using image data to analyze an image

Patent metadata

Field	Value
Publication number	US-12597271-B2
Application number	US-202318303460-A
Country	US
Kind code	B2
Filing date	Apr 19, 2023
Priority date	Sep 23, 2022
Publication date	Apr 7, 2026
Grant date	Apr 7, 2026

Abstract

Systems and methods for training and executing machine learning models to generate lane index values are disclosed. A method includes identifying a set of image data captured by at least one autonomous vehicle when the at least autonomous vehicle is positioned in a lane of a roadway and respective ground truth localization data; determining a plurality of lane index values for the set of image data based on the ground truth localization data; labeling the set of image data with the plurality of lane index values, the lane index values representing a number of lanes from a leftmost or rightmost lane to the lane in which the at least one autonomous vehicle was positioned; and training, using the labeled set of image data, a plurality of machine learning models that generate a left lane index value and a right lane index value as output.

First claim

What is claimed is:

1 . A method, comprising:
identifying, by one or more processors coupled to non-transitory memory, (i) a set of image data captured by at least one autonomous vehicle when the at least one autonomous vehicle was positioned in a lane of a roadway, and (ii) respective ground truth location data of the at least one autonomous vehicle; determining, by the one or more processors, a plurality of lane index values for the set of image data based on the ground truth location data; labeling, by the one or more processors, the set of image data with the plurality of lane index values, the plurality of lane index values representing a number of lanes from a leftmost or rightmost lane to the lane in which the at least one autonomous vehicle was positioned; and training, by the one or more processors, using the labeled set of image data, a plurality of machine learning models that generate a left lane index value and a right lane index value as output, wherein the plurality of machine learning models includes a first machine learning model and a second machine learning model, the left lane index value exclusively generated by the first machine learning model as output and the right lane index value exclusively generated by the second machine learning model as output; wherein at least one controller of the at least one autonomous vehicle is configured to control operation of the vehicle based on the trained plurality of machine learning models.

2 . The method of claim 1 , wherein the ground truth location data includes data derived from a high-definition (HD) map.

3 . The method of claim 2 , wherein a plurality of lane indications of the set of image data are defined at least in part as a feature on a raster layer of the high-definition (HD) map.

4 . The method of claim 1 , further comprising evaluating, by the one or more processors, the plurality of machine learning models based on a second set of image data depicting a field of view of the at least one autonomous vehicle.

5 . The method of claim 1 , wherein the plurality of machine learning models each comprise a plurality of neural network layers.

6 . The method of claim 1 , further comprising providing, by the one or more processors, the plurality of machine learning models to an autonomous vehicle for execution during operation of the autonomous vehicle.

7 . A system, comprising:
one or more processors coupled to a non-transitory memory, wherein the one or more processors are configured to:
identify (i) a set of image data captured by at least one autonomous vehicle when the at least one autonomous vehicle was positioned in a lane of a roadway, and (ii) respective ground truth location data of the at least one autonomous vehicle;
determine a plurality of lane index values for the set of image data based on the ground truth location data;
label the set of image data with the plurality of lane index values, the plurality of lane index values representing a number of lanes from a leftmost or rightmost lane to the lane in which the at least one autonomous vehicle was positioned; and
train, using the labeled set of image data, a plurality of machine learning models that generate a left lane index value and a right lane index value as output, wherein the plurality of machine learning models includes a first machine learning model and a second machine learning model, the left lane index value exclusively generated by the first machine learning model as output and the right lane index value exclusively generated by the second machine learning model as output,
wherein at least one controller of the at least one autonomous vehicle is configured to control operation of the vehicle based on the trained plurality of machine learning models.

8 . The system of claim 7 , wherein the one or more processors are further configured to evaluate the plurality of machine learning models based on a second set of image data depicting a field of view of the at least one autonomous vehicle.

9 . The system of claim 7 , wherein the ground truth location data includes data derived from a high-definition (HD) map.

10 . The system of claim 9 , wherein a plurality of lane indications of the set of image data are defined at least in part as a feature on a raster layer of the high-definition (HD) map.

11 . The system of claim 7 , wherein the plurality of machine learning models each comprise a plurality of neural network layers.

12 . The system of claim 7 , wherein the one or more processors are further configured to provide the plurality of machine learning models to an autonomous vehicle for execution during operation of the autonomous vehicle.

13 . A method, comprising:
identifying, by one or more processors of an autonomous vehicle, image data indicative of a field of view from the autonomous vehicle when the autonomous vehicle is positioned in a lane of a multi-lane roadway; executing, by the one or more processors, a plurality of machine learning models using the image data as input to generate a left lane index value and a right lane index value of the lane, wherein the plurality of machine learning models includes a first machine learning model and a second machine learning, the left lane index value exclusively generated by the first machine learning model as output and the right lane index value exclusively generated by as output; and localizing, by the one or more processors, the autonomous vehicle based at least on the left lane index value and the right lane index value.

14 . The method of claim 13 , wherein the plurality of machine learning models each comprise a plurality of neural network layers.

15 . The method of claim 13 , further comprising providing, by the one or more processors, the plurality of machine learning models to an autonomous vehicle for execution during operation of the autonomous vehicle.

16 . The method of claim 13 , further comprising evaluating, by the one or more processors, the plurality of machine learning models based on a second set of image data depicting a field of view of the autonomous vehicle.

Assignees

• Torc Robotics Inc

Inventors

• Chilton Ryan
• Pullagurla Harish
• Stamenkovich Joseph

Classifications

• B60W2300/14

  Tractor-trailers, i.e. combinations of a towing vehicle and one or more towed vehicles, e.g. caravans; Road trains · CPC title

• B60W60/001

  Planning or execution of driving tasks · CPC title

• B60W2556/40

  High definition maps · CPC title

• B60W2552/53

  Road markings, e.g. lane marker or crosswalk · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title