Vehicle localization based on lane templates

Patent metadata

Field	Value
Publication number	US-12561989-B2
Application number	US-202418650660-A
Country	US
Kind code	B2
Filing date	Apr 30, 2024
Priority date	Nov 7, 2022
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

Methods, systems, and non-transitory computer-readable media are configured to perform operations comprising determining lane detection data and object detection data associated with an environment; determining a lane template based on the lane detection data; and generating localization data that identifies a location of an object in the environment based on the lane template and the object detection data.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
capturing image data by a camera of a vehicle; based on a machine learning model trained to classify detection data into a lane template, selecting, by a computing system, based on detection data and the machine learning model, a lane template from a set of predefined lane templates, the detection data associated with an environment and including detected lane boundaries in the image data captured by the vehicle, the detection data based on the image data; determining, by the computing system, a first location of the vehicle in the environment and a second location of an object relative to the vehicle based on the detection data; and generating, by the computing system, localization data that identifies the first location of the vehicle and the second location of the object on the lane template.

2 . The computer-implemented method of claim 1 , wherein determining the first location of the vehicle in the environment and the second location of the object relative to the vehicle comprises:
detecting, by the computing system, lane markings in the environment based on the detection data; and determining, by the computing system, a lane where the vehicle is located in the environment.

3 . The computer-implemented method of claim 1 , wherein determining the first location of the vehicle in the environment and the second location of the object relative to the vehicle comprises:
detecting, by the computing system, the object in image data associated with the environment; and determining, by the computing system, the second location of the object relative to the vehicle based on a third location of the object in the image data.

4 . The computer-implemented method of claim 1 , wherein generating the localization data comprises:
providing, by the computing system, a first indication on the lane template based on the first location of the vehicle in the environment; and providing, by the computing system, a second indication on the lane template, wherein the second indication is indicative of where the object is in the environment based on the second location of the object relative to the vehicle.

5 . The computer-implemented method of claim 1 , wherein determining the first location of the vehicle in the environment and the second location of the object relative to the vehicle is based on a second machine learning model, wherein a first indication that corresponds with the first location is provided on the lane template based on the second machine learning model, and wherein a second indication that corresponds with the second location is provided on the lane template based on the second machine learning model.

6 . The computer-implemented method of claim 5 , further comprising:
generating, by the computing system, a first instance of training data for the second machine learning model based on the localization data and the detection data.

7 . The computer-implemented method of claim 6 , further comprising:
simulating, by the computing system, a variation of the first instance of training data based on a modification of the second location of the object on the lane template; and generating, by the computing system, a second instance of training data for the second machine learning model based on the variation.

8 . The computer-implemented method of claim 6 , further comprising:
comparing, by the computing system, an output of the second machine learning model with the lane template populated with the first location of the vehicle and the second location of the object.

9 . The computer-implemented method of claim 1 , wherein determining the first location of the vehicle in the environment and the second location of the object relative to the vehicle comprises:
indicating, by the computing system, the object in the detection data based on a bounding box.

10 . The computer-implemented method of claim 1 , further comprising:
providing, by the computing system, the localization data to a perception system, wherein the localization data causes the perception system to locate the vehicle and the object in the environment.

11 . A system comprising:
at least one processor; and a memory storing instructions that, when executed, cause the system to perform operations comprising:
capturing image data by a camera of a vehicle;
based on a machine learning model trained to classify detection data into a lane template, selecting, based on detection data and the machine learning model, a lane template from a set of predefined lane templates, the detection data associated with an environment and including detected lane boundaries in the image data captured by the vehicle, the detection data based on the image data;
determining a first location of the vehicle in the environment and a second location of an object relative to the vehicle based on the detection data; and
generating localization data that identifies the first location of the vehicle and the second location of the object on the lane template.

12 . The system of claim 11 , wherein determining the first location of the vehicle in the environment and the second location of the object relative to the vehicle comprises:
detecting lane markings in the environment based on the detection data; and determining a lane where the vehicle is located in the environment.

13 . The system of claim 11 , wherein determining the first location of the vehicle in the environment and the second location of the object relative to the vehicle comprises:
detecting the object in image data associated with the environment; and determining the second location of the object relative to the vehicle based on a third location of the object in the image data.

14 . The system of claim 11 , wherein generating the localization data comprises:
providing a first indication on the lane template based on the first location of the vehicle in the environment; and providing a second indication on the lane template, wherein the second indication is indicative of where the object is in the environment based on the second location of the object relative to the vehicle.

15 . The system of claim 11 , wherein determining the first location of the vehicle in the environment and the second location of the object relative to the vehicle is based on a second machine learning model, wherein a first indication that corresponds with the first location is provided on the lane template based on the second machine learning model, and wherein a second indication that corresponds with the second location is provided on the lane template based on the second machine learning model.

16 . A non-transitory computer-readable storage medium including instructions that, when executed, cause a computing system to perform operations comprising:
capturing image data by a camera of a vehicle; based on a machine learning model, selecting trained to classify detection data into a lane template, based on detection data and the machine learning model, a lane template from a set of predefined lane templates, the detection data associated with an environment and including detected lane boundaries in the image data captured by the vehicle, the detection data based on the image data; determining a first location of the vehicle in the environment and a second location of an object relative to the vehicle based on the detection data; and generating localization data that identifies the first location of the vehicle and the second location of the object on the lane template.

17 . The

Assignees

• Plusai Inc

Inventors

• Zhou Mianwei
• Chen Yibo
• Bhatia Abhishek

Classifications

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06V10/7747

  Organisation of the process, e.g. bagging or boosting · CPC title

• G06V20/588 Primary

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title