Localization with diverse dataset for autonomous vehicles

Patent metadata

Field	Value
Publication number	US-11820397-B2
Application number	US-202017018499-A
Country	US
Kind code	B2
Filing date	Sep 11, 2020
Priority date	Nov 16, 2019
Publication date	Nov 21, 2023
Grant date	Nov 21, 2023

Abstract

A computer-implemented method for localizing a vehicle can include accessing, by a computing system comprising one or more computing devices, a machine-learned retrieval model that has been trained using a ground truth dataset comprising a plurality of pre-localized sensor observations. Each of the plurality of pre-localized sensor observations has a predetermined pose value associated with a previously obtained sensor reading representation. The method also includes obtaining, by the computing system, a current sensor reading representation obtained by one or more sensors located at the vehicle. The method also includes inputting, by the computing system, the current sensor reading representation into the machine-learned retrieval model. The method also includes receiving, by the computing system and from the machine-learned retrieval model, a determined current pose value for the vehicle based at least in part on one or more of the pre-localized sensor observations determined to be a closest match to the current sensor reading representation. The determined current pose value has an accuracy of within about one meter.

First claim

What is claimed is:

1. A computer-implemented method for localizing a vehicle, the computer-implemented method comprising:
accessing one or more computing devices, a machine-learned retrieval model that has been trained using a ground truth dataset comprising a plurality of pre-localized sensor observations, respective pre-localized sensor observations having a predetermined pose value associated with a previously obtained sensor reading representation;
obtaining a current sensor reading representation obtained by one or more sensors located at the vehicle;
inputting the current sensor reading representation into the machine-learned retrieval model;
receiving from the machine-learned retrieval model, a determined current pose value for the vehicle based at least in part on one or more pre-localized sensor observations of the plurality of pre-localized sensor observations determined to be a closest match to the current sensor reading representation, wherein the determined current pose value has an accuracy of within about one meter;
obtaining a current GPS estimate for the vehicle; and
determining a subset of the plurality of pre-localized sensor observations that are within a predetermined threshold distance of the current GPS estimate for the vehicle; and
wherein the one or more of the pre-localized sensor observations determined to be the closest match to the current sensor reading are determined from the subset of the plurality of pre-localized sensor observations that are within the predetermined threshold distance of the current GPS estimate for the vehicle.

2. The computer-implemented method of claim 1 , wherein the determined current pose value comprises the predetermined pose value of the one or more pre-localized sensor observations determined to be the closest match to the current sensor reading representation.

3. The computer-implemented method of claim 1 , wherein each pre-localized sensor observation and the current sensor reading representation comprises a vector of features determined from a respective sensor reading.

4. The computer-implemented method of claim 1 , wherein the one or more sensors comprises at least one of one or more LIDAR sensors or one or more cameras.

5. The computer-implemented method of claim 1 , wherein:
the one or more sensors comprises a LIDAR sensor configured to obtain a LIDAR point cloud; and
the current sensor reading representation is determined from a multi-channel bird's eye view representation of the LIDAR point cloud that is discretized into a plurality of voxels.

6. The computer-implemented method of claim 1 , wherein the machine-learned retrieval model is trained with respect to a triplet loss function determined for each pre-localized sensor observation in the ground truth dataset, the triplet loss function defined in terms of a positive input, an anchor input, and a negative input, wherein a first threshold for comparing the positive input to the anchor input is less than a second threshold for comparing the negative input to the anchor input.

7. The computer-implemented method of claim 6 , wherein each of the positive input, the negative input, and the anchor input have an associated heading angle, and wherein the heading angles for each of the positive input, the negative input and the anchor input are within a predetermined angular range.

8. The computer-implemented method of claim 6 , wherein the positive input and the negative input are captured along at least one different trip than a trip along which the anchor input is captured.

9. The computer-implemented method of claim 1 , wherein the determined current pose value has an accuracy within about 10 centimeters.

10. The computer-implemented method of claim 1 , wherein the ground truth dataset comprises pre-localized sensor observations taken under differing conditions of at least one of weather, season, illumination, construction, occlusion, or dynamic objects.

11. The computer-implemented method of claim 10 , wherein the differing conditions comprise at least one of LIDAR occlusion, image occlusion, temperature, cloud cover, precipitation intensity, sun angle over horizon, visibility, UV conditions, precipitation type, or trip.

12. The computer-implemented method of claim 1 , wherein the pre-localized sensor observations are localized by vehicle dynamics and LIDAR registration against a dense scan of a region.

13. The computer-implemented method of claim 1 , wherein the ground truth dataset is annotated with granular labels from at least one of historical weather data, historical astronomical data, or degree of occlusion.

14. The computer-implemented method of claim 13 , further comprising filtering, from the plurality of candidate features, one or more of the plurality of ground truth features based at least in part on the granular labels.

15. A computer-implemented method for generating a ground truth dataset, the computer-implemented method comprising:
obtaining, by a computing system comprising one or more computing devices, a dense scan of a region, the dense scan comprising one or more sensor observations descriptive of a plurality of ground truth features;
obtaining, by the computing system, a plurality of dataset sensor observations of the region, the plurality of dataset sensor observations of the region descriptive of the plurality of ground truth features,
localizing, based at least in part on vehicle dynamics and LIDAR registration, the plurality of dataset sensor observations against the dense scan to determine a pose of each of the plurality of dataset sensor observations; and
providing, by the computing system, the plurality of dataset sensor observations and the pose of each of the plurality of dataset sensor observations for retrieval in a ground truth dataset, wherein the ground truth dataset is configured for access by an autonomous vehicle to subsequently determine real-time localization.

16. The computer-implemented method of claim 15 , wherein the dense scan comprises a LIDAR scan.

17. The computer-implemented method of claim 15 , wherein the plurality of dataset sensor observations are captured under differing conditions, and wherein the differing conditions comprise at least one of LIDAR occlusion, image occlusion, temperature, cloud cover, precipitation intensity, sun angle over horizon, visibility, UV conditions, precipitation type, or trip.

18. The computer-implemented method of claim 15 , wherein the method further comprises:
accessing, by the computing system, a machine-learned retrieval model that has been trained using the ground truth dataset;
obtaining, by the computing system, a current sensor reading representation obtained by one or more sensors located at the vehicle;
inputting, by the computing system, the current sensor reading representation into the machine-learned retrieval model; and
receiving, by the computing system and from the machine-learned retrieval model, a determined current pose value for the vehicle based at least in part on one or more of the pre-localized sensor observations determined to be a closest match to the current sensor reading representation, wherein the determined current pose value has an accuracy of within about one meter.

19. The computer-implemented method of claim 15 , wherein the ground truth dataset is annotated with granular labels from at least one of historical weather data, historical astronomical data, or degree of occlusion.

20. A computing system for localizing a vehicle comprising:
 one or more processors; and
 one

Assignees

• Uatc Llc

Inventors

• Martinez Covarrubias Julieta
• Urtasun Raquel
• Wang Shenlong
• Barsan Ioan Andrei
• Mattyus Gellert Sandor
• Doubov Alexandre
• Fan Hongbo

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• B60W60/001 Primary

  Planning or execution of driving tasks · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06T15/08

  Volume rendering · CPC title