Correction of sensor data alignment and environment mapping

Patent metadata

Field	Value
Publication number	US-12461524-B2
Application number	US-202318111463-A
Country	US
Kind code	B2
Filing date	Feb 17, 2023
Priority date	Nov 26, 2019
Publication date	Nov 4, 2025
Grant date	Nov 4, 2025

Abstract

Generating a map associated with an environment may include collecting sensor data received from one or more vehicles and generating a set of links to align the sensor data. A mesh representation of the environment may be generated from the aligned sensor data. A system may determine a proposed link to add, a proposed link deletion, and/or a proposed link alteration, and receive a modification comprising instructions to add, delete, or modify a link. Responsive to receiving a modification, the system may re-align a window of sensor data associated with the modification. The modification and/or sensor data associated therewith may be collected as training data for a machine learning model, which may be trained to generate link modification proposals and/or determine sensor data that may be associated with a poor sensor data alignment.

First claim

What is claimed is:

1 . A method comprising:
receiving sensor data associated with an environment; determining, based at least in part on the sensor data and a first link, a first map and a first trajectory associated with a vehicle, the first trajectory comprising a plurality of poses and the first link comprising a first indication that a first portion of the sensor data associated with a first pose of the first trajectory corresponds to a second portion of the sensor data associated with a second pose, wherein determining the first link comprises:
determining, based on the first portion, the first pose,
determining, based on the second portion, the second pose,
determining a distance based on the first pose and the second pose,
determining that the distance falls below a threshold, and
based at least in part on determining that the distance falls below the threshold, generating the first link in association with the first portion and the second portion;
receiving, from a user, an instruction for a modification comprising at least one of adding a second link or deleting the first link, the second link comprising a second indication that a third portion of sensor data associated with a third pose of the plurality of poses corresponds to a fourth portion of sensor data associated with a fourth pose of the plurality of poses; determining, based at least in part on the sensor data and the modification, a second map and a second trajectory; and transmitting the second map to an additional vehicle, wherein the additional vehicle is configured to be controlled based at least in part on the second map.

2 . The method of claim 1 , wherein:
the sensor data comprises first sensor data and second sensor data, wherein the first sensor data is associated with a first sensor of a first system and the second sensor data is associated with a second sensor of a second system.

3 . The method of claim 1 , wherein:
the modification comprises adding the second link, determining at least one of the second map or the second trajectory comprises, based on determining that the modification comprises adding the second link, determining a window of sensor data associated with the modification, determining the window of sensor data is based on a threshold distance associated with at least one of the third portion or the fourth portion, and determining the second map comprises re-determining part of the first map associated with a portion of the environment corresponding to the window of sensor data.

4 . The method of claim 3 , wherein determining the window of sensor data comprises determining a subset of the sensor data that is within:
a threshold distance of a position associated with at least one of the first link or the second link, or a threshold time of a time associated with at least one of the first link or the second link.

5 . The method of claim 3 , wherein:
determining to associate the first portion and the second portion by the first link is based at least in part on at least one of a covariance, sampling rate, or scalar associated with the first link; and the modification comprises at least one of adding the second link, deleting the first link, or modifying the first link by altering at least one of the covariance, the sampling rate, or the scalar.

6 . The method of claim 1 , wherein:
based at least in part on receiving the modification, the modification, the first portion, and the second portion are added to a training data set; and the method further comprises training a machine-learning model based at least in part on the training data set, the machine-learning model trained to identify one or more second modifications to add a third link or remove the third link.

7 . A system comprising:
one or more processors; and a memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising: receiving sensor data associated with an environment; determining, based at least in part on the sensor data and a first link, a first map and a first trajectory associated with a vehicle, the first trajectory comprising a plurality of poses and the first link comprising a first indication that a first portion of the sensor data associated with a first pose of the first trajectory corresponds to a second portion of the sensor data associated with a second pose, wherein determining the first link comprises:
determining, based on the first portion, the first pose,
determining, based on the second portion, the second pose,
determining a distance based on the first pose and the second pose,
determining that the distance falls below a threshold, and
based at least in part on determining that the distance falls below the threshold, generating the first link in association with the first portion and the second portion;
receiving, from a user, an instruction for a modification comprising at least one of adding a second link or deleting the first link, the second link comprising a second indication that a third portion of sensor data associated with a third pose of the plurality of poses corresponds to a fourth portion of sensor data associated with a fourth pose of the plurality of poses; determining, based at least in part on the sensor data and the modification, a second map and a second trajectory; and transmitting the second map to an additional vehicle, wherein the additional vehicle is configured to be controlled based at least in part on the second map.

8 . The system of claim 7 , wherein:
the sensor data comprises first sensor data and second sensor data, wherein the first sensor data is associated with a first sensor of a first system and the second sensor data is associated with second sensor a second system.

9 . The system of claim 7 , wherein:
the modification comprises adding the second link, determining at least one of the second map or the second trajectory comprises, based on determining that the modification comprises adding the second link, determining a window of sensor data associated with the modification, determining the window of sensor data is based on a threshold distance associated with at least one of the third portion or the fourth portion, and determining the second map comprises re-determining part of the first map associated with a portion of the environment corresponding to the window of sensor data.

10 . The system of claim 9 , wherein determining the window of sensor data comprises determining a subset of the sensor data that is within:
a threshold distance of a position associated with at least one of the first link or the second link, or a threshold time of the position or time associated with the at least one of the first link or the second link.

11 . The system of claim 9 , wherein:
determining to associate the first portion and the second portion by the first link is based at least in part on at least one of a covariance, sampling rate, or scalar associated with the first link; and the modification comprises at least one of adding the second link, deleting the first link, or modifying the first link by altering at least one of the covariance, the sampling rate, or the scalar.

12 . The system of claim 7 , wherein:
based at least in part on receiving the modification, the modification, the first portion, and the second portion are added to a training data set; and the operations further comprise training a machine-learning model based at least in part on the training data set, the machine-learning model trained to identify one or more second modifications to add third link or remove the third link.

Assignees

• Zoox Inc

Inventors

• Liu Chengzhi Winston
• Adams Derek

Classifications

• G01C21/3602

  Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera · CPC title

• G01C21/3804

  Creation or updating of map data · CPC title

• G01C21/3815 Primary

  Road data · CPC title

• G01C21/387

  Organisation of map data, e.g. version management or database structures · CPC title

• G01C21/3676

  Overview of the route on the road map · CPC title