Method and system for localization of an autonomous vehicle in real time

Patent metadata

Field	Value
Publication number	US-11262759-B2
Application number	US-201916655132-A
Country	US
Kind code	B2
Filing date	Oct 16, 2019
Priority date	Oct 16, 2019
Publication date	Mar 1, 2022
Grant date	Mar 1, 2022

Abstract

A system and method for localization of an autonomous vehicle based on 3D point cloud are disclosed. The system includes a processor and a memory coupled to the processor, the memory tangibly storing thereon executable instructions that, when executed by the processor, cause the processor to: receive an initial position of a vehicle; receive a data file representative of a three-dimensional (3D) point cloud from a LIDAR scanning subsystem of the vehicle; divide the 3D point cloud into a plurality of tiles and compute a plurality of key points for each tile; generate an estimated low frequency 2D position and yaw angle for the vehicle based on a particle filter process and a dynamically downloaded 2D reference map; and generate a position and orientation of the vehicle.

First claim

The invention claimed is:

1. A method for localization of an autonomous vehicle, the method comprising:
receiving a three-dimensional (3D) point cloud of an environment surrounding the autonomous vehicle captured by a light detection and ranging (LIDAR) scanning system of the autonomous vehicle;
generating a two-dimensional (2D) image of the environment based on the 3D point cloud, the 2D image comprising a plurality of pixels, wherein each respective pixel of the plurality of pixels has at least one of: a first channel indicative of a vertical distribution value of the pixel and a second channel indicative of an intensity value of the pixel;
estimating a position of the vehicle by applying a particle filtering process on the 2D image;
retrieving odometry data from a wheel odometry unit of the autonomous vehicle, the odometry data comprising rotation data indicative of rotation of the autonomous vehicle and translation data indicative of a translation of the autonomous vehicle;
generating a location of the autonomous vehicle, the location of the autonomous vehicle comprising a final position of the vehicle determined based on the odometry data and the estimated position of the autonomous vehicle and an elevation of the vehicle determined based on the final position of the autonomous vehicle and a 2D reference map of the environment.

2. The method of claim 1 , further comprising:
estimating a yaw angle value of the autonomous vehicle based on the 2D image;
determining a final yaw angle of the autonomous vehicle based on the odometry data and the estimated yaw angle value of the vehicle; and
wherein the location of the autonomous vehicle further comprises the final yaw angle of the vehicle.

3. The method of claim 1 , further comprising retrieving further odometry data from an inertial measurement unit (IMU) of the autonomous vehicle, the further odometry data comprising three-axis angular velocity of the autonomous vehicle and three-axis acceleration of the autonomous vehicle and wherein the final position of the autonomous vehicle is further determined based on the further odometry data retrieved from the IMU.

4. The method of claim 3 , further comprising:
determining a final roll value and a final pitch of the vehicle based on the data retrieved from the IMU; and
wherein the location of the vehicle further comprises the final roll value and the final pitch value of the autonomous vehicle.

5. The method of claim 1 , wherein the final position of the autonomous vehicle is determined using an Extended Kalman filter (EKF).

6. The method of claim 2 , wherein the final yaw angle value of the autonomous vehicle is determined using an Extended Kalman filter (EKF).

7. The method of claim 1 , wherein the 3D point cloud comprises a plurality of data points, wherein each data point has an intensity value and a set of coordinates, and wherein generating a two-dimensional (2D) image of the environment based on the 3D point cloud comprises;
dividing the 3D point cloud along a plane into a plurality of titles;
dividing each tile into a plurality of cells, wherein each cell has a length T along a z-axis of a coordinate system, the length T being defined by an upper bound and a lower bound in parallel with a ground surface of the 3D point cloud, and wherein each of the plurality of cells has a 2D position [x k L , y k L ] based on the coordinate system and a yaw angle value θ L ;
for each cell in each tile:
dividing the cell along the length T into a plurality of s blocks of equal heights;
for each of the plurality of s blocks, assigning a value of 1 when the block contains at least one data point from the 3D point cloud, and assigning a value of 0 when the block does not contain any data point from the 3D point cloud, such that an s-bit binary sequence is generated for each of the plurality of cells, and each binary digit in the s-bit binary sequence is equivalent to the assigned value of a corresponding block in the plurality of s blocks of the cell;
converting the s-bit binary sequence into a corresponding decimal number and save the corresponding decimal number in the first channel of a data set representing the cell;
when the cell is determined to be flat, determining an intensity value based on a maximum intensity value of one or more data points within the cell and saving the intensity value in the second channel of the data set representing the cell;
when the cell is determined to be not flat, assigning a value of 0 in the second channel of the data set representing the cell;

for each tile, generating a 2D image of the 3D point cloud based on the data set representing each cell.

8. The method of claim 7 , further comprising:
sorting the pixels of the 2D image based on the value of the first channel of each pixel in the 2D image, and selecting from the sorted pixels the first m hundred pixels as the plurality of key points for vertical distribution for the 2D image, wherein each key point has a vertical distribution value v i vert , i=1, 2, . . . , m, v i vert equal to the respective value of the respective first channel of the key point;
sorting from largest to smallest, the plurality of pixels based on the respective value of the respective second channel of each of the plurality of pixels, and select from the sorted pixels the first l hundred pixels as the plurality of key points for intensity for the 2D image, wherein each key point has an intensity value v i vert , i=1, 2, . . . , l, v i inte equal to the respective value of the respective second channel of the key point;
for each key point for vertical distribution in the 2D image, calculating and storing an importance gain value for the key point for vertical distribution;
calculating and storing the vertical distribution weight value for the 2D image based on the importance gain value of at least one key point for vertical distribution;
calculating and storing the intensity weight value for the 2D image based on at least one key point for intensity; and
updating one or both of the vertical distribution weight value and the intensity weight value based on at least an information entropy value of one or both of the vertical distribution weight value and the intensity weight value.

9. The method of claim 8 , wherein for each key point in the 2D image, the importance gain value for the key point for vertical distribution is represented by a value α ij is determined in accordance with the following equation:
α ij =−N ij +N max +k

wherein i denotes a row number of the key point within the 2D image, j denotes a column number of the key point within the 2D image, N max denotes a total number of points in the 2D image that has the highest points in the 2D image, and N min denotes a total number of points in the 2D image that has the least non-zero points in the 2D image, and k is a positive integer.

10. The method of claim 9 , wherein the vertical distribution weight value is represented by w i vert and determined by the equation:
w i vert =Σ k=1 n α k ·μ k vert ·v k vert ,

wherein k is an integer from 1 to n, n is a total number m of key points for vertical distribution in the respective 2D image, v k vert is the vertical distribution value of the key point, α k is the importance gain value for the key point for vertical distribution, and μ k vert is a corresponding vertical distribution value for the key point from the 2D reference map; and
wherein for each key point in the 2D image, the intensity weight value is represented by w i inte as defined by the equation:
w i inte =Σ k=1 n μ k inte · k inte ,

wherein k is an integer from 1 to n, n is a total number l of

Assignees

• Ren Yuan
• Liu Bingbing
• Huawei Tech Co Ltd

Inventors

• Ren Yuan
• Liu Bingbing

Classifications

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T7/73

  using feature-based methods · CPC title