Autonomous vehicle localization using a Lidar intensity map

Patent metadata

Field	Value
Publication number	US-11726208-B2
Application number	US-201916437827-A
Country	US
Kind code	B2
Filing date	Jun 11, 2019
Priority date	Jun 15, 2018
Publication date	Aug 15, 2023
Grant date	Aug 15, 2023

Abstract

Aspects of the present disclosure involve systems, methods, and devices for autonomous vehicle localization using a Lidar intensity map. A system is configured to generate a map embedding using a first neural network and to generate an online Lidar intensity embedding using a second neural network. The map embedding is based on input map data comprising a Lidar intensity map, and the Lidar sweep embedding is based on online Lidar sweep data. The system is further configured to generate multiple pose candidates based on the online Lidar intensity embedding and compute a three-dimensional (3D) score map comprising a match score for each pose candidate that indicates a similarity between the pose candidate and the map embedding. The system is further configured to determine a pose of a vehicle based on the 3D score map and to control one or more operations of the vehicle based on the determined pose.

First claim

What is claimed is:

1. An autonomous vehicle (AV) system configured to control a vehicle, the system comprising:
one or more processors of a machine; and
a machine-storage medium storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising:
generating, using a first embedding function, an intensity map embedding based on a Lidar intensity map, the Lidar intensity map comprising a map image encoded with Lidar intensity data, the intensity map embedding comprising a representation of the Lidar intensity map computed by the first embedding function, the map image comprising a birds-eye view (BEV) image of an environment;
generating, using a second embedding function, an online Lidar intensity embedding based on an online Lidar intensity image, the online lidar intensity image comprising a BEV rasterized image generated by aggregating point data output by a Lidar system during operation of the vehicle, the aggregating using IMU data and wheel odometer read information, the online Lidar intensity embedding comprising a representation of the online Lidar intensity image computed by the second embedding function, the point data comprising multiple point clouds;
transforming the online Lidar intensity embedding into a coordinate frame of the intensity map embedding;
generating a plurality of pose candidates based on the online Lidar intensity embedding;
computing a three-dimensional (3D) score map based on a comparison of the intensity map embedding with each pose candidate in the plurality of pose candidates, the 3D score map comprising a plurality of match scores, the plurality of match scores comprising a match score for a pose candidate in the plurality of pose candidates, the match score for the pose candidate indicating a similarity between the pose candidate and the map embedding;
determining a pose of the vehicle based on the 3D score map, the pose of the vehicle corresponding to the pose candidate, the pose of the vehicle comprising a longitude, a latitude, and a heading; and
controlling one or more operations of the vehicle based on the pose.

2. The AV system of claim 1 , wherein the operations further comprise:
computing a localization score array based on the 3D score map, the localization score array comprising a plurality of localization scores generated based on the plurality of match scores, the plurality of localization scores including a localization score for the pose candidate.

3. The AV system of claim 2 , wherein the computing of the localization score array comprises computing an element-wise product of the 3D score map, a vehicle motion model, and a GPS observation model score map, the vehicle motion model encoding an agreement between an inferred velocity and a velocity sensed by an IMU and one or more wheel encoders, the GPS observation model score map encoding an agreement between a hypothesized vehicle pose and a current GPS observation.

4. The AV system of claim 3 , wherein the operations further comprise computing the motion model based on a previous vehicle pose and vehicle dynamics.

5. The AV system of claim 2 , wherein the determining of the pose of the vehicle is based on the localization score for the pose candidate.

6. The AV system of claim 1 , wherein the second embedding function computes the representation of the online Lidar intensity image by performing operations comprising:
identifying a portion of the online Lidar intensity image comprising one or more unreliable localization cues; and
masking out the portion of the online Lidar intensity image comprising the one or more unreliable localization cues.

7. The AV system of claim 1 , wherein the generating of the plurality of pose candidates based on the online Lidar intensity embedding comprises performing multiple rotations of the online Lidar intensity embedding.

8. The AV system of claim 1 , wherein the computing of the 3D score map comprises convolving the intensity map embedding with each pose candidate in the plurality of pose candidates.

9. The AV system of claim 8 , wherein the convolving of the intensity map embedding with each pose candidate comprises performing a convolution of the intensity map embedding with the online Lidar intensity image embedding for each pose candidate in a transformed domain.

10. The AV system of claim 1 , wherein the Lidar intensity map further includes a height map containing a height of each point in the Lidar intensity map with respect to a coordinate frame.

11. A method comprising:
generating, using a first neural network, an intensity map embedding based on a Lidar intensity map, the Lidar intensity map comprising a map image encoded with Lidar intensity data, the intensity map embedding comprising a representation of the Lidar intensity map computed by the first neural network, the map image comprising a birds-eye view (BEV) image of an environment;
generating, using a second neural network, an online Lidar intensity embedding based on an online Lidar intensity image, the online lidar intensity image comprising a BEV rasterized image generated by aggregating point data output by a Lidar system during operation of a vehicle, the aggregating using IMU data and wheel odometer read information, the online Lidar intensity embedding comprising a representation of the online Lidar intensity image computed by the second neural network;
transforming the online Lidar intensity embedding into a coordinate frame of the intensity map embedding;
generating a plurality of pose candidates based on the online Lidar intensity embedding;
computing a three-dimensional (3D) score map based on a comparison of the intensity map embedding with each pose candidate in the plurality of pose candidates, the 3D score map comprising a plurality of match scores, the plurality of match scores comprising a match score for a pose candidate in the plurality of pose candidates, the match score for the pose candidate indicating a similarity between the pose candidate and the intensity map embedding;
determining a pose of the vehicle based on the 3D score map, the pose of the vehicle corresponding to the pose candidate, the pose of the vehicle comprising a longitude, a latitude, and a heading; and
controlling one or more operations of the vehicle based on the determined pose.

12. The method of claim 11 , further comprising:
computing a localization score array based on the 3D score map, the localization score array comprising a plurality of localization scores generated based on the plurality of match scores, the plurality of localization scores including a localization score for the pose candidate.

13. The method of claim 12 , wherein the computing of the localization score array comprises computing an element-wise product of the 3D score map, a vehicle motion model, and a GPS observation model score map, the vehicle motion model encoding an agreement between an inferred velocity and a velocity sensed by an IMU and one or more wheel encoders, the GPS observation model score map encoding an agreement between a hypothesized vehicle pose and a current GPS observation.

14. The method of claim 12 , wherein the determining of the pose of the vehicle is based on the localization score for the pose candidate.

15. The method of claim 11 , wherein the second neural network computes the representation of the online Lidar intensity image by performing operations comprising:
identifying a portion of the online Lidar intensity image comprising one or more unreliable localization cues; and
masking out the portion of the online Lidar intensity image comprising the o

Assignees

• Uatc Llc

Inventors

• Wang Shenlong
• Pokrovsky Andrei
• Sotil Raquel Urtasun
• Bârsan Ioan Andrei

Classifications

• G05D1/024

  in combination with a laser (lasers per se H01S) · CPC title

• G05D1/0274

  using mapping information stored in a memory device (navigation using map-matching G01C21/30) · CPC title

• G01S17/42 Primary

  Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S17/46) · CPC title

• G01S7/4861

  Circuits for detection, sampling, integration or read-out · CPC title

• G01S17/931

  of land vehicles · CPC title