Object detection using local (ground-aware) adaptive region proposals on point clouds

Patent metadata

Field	Value
Publication number	US-11733353-B2
Application number	US-202016781747-A
Country	US
Kind code	B2
Filing date	Feb 4, 2020
Priority date	Nov 14, 2019
Publication date	Aug 22, 2023
Grant date	Aug 22, 2023

Abstract

Embodiments include a method for object detection in a Light Detection And Ranging (LiDAR) point cloud, the method comprising: placing, by a navigation system, a plurality of anchor points in a two-dimensional Bird's Eye View (BEV) of spatial points represented in a segmented ground surface representation of objects detected by a LiDAR system; extracting, by the navigation system, one or more features from the two-dimensional BEV of the spatial points; proposing, by the navigation system, one or more regions of the two-dimensional BEV of the spatial points for object detection; and performing, by the navigation system, object detections on anchor points of the plurality of anchor points in the proposed one or more regions of the two-dimensional BEV of the spatial points.

First claim

What is claimed is:

1. A method for object detection in a Light Detection And Ranging (LiDAR) point cloud, the method comprising:
placing, by a navigation system, a plurality of anchor points in a two-dimensional Bird's Eye View (BEV) of spatial points represented in a segmented ground surface representation of objects detected by a LiDAR system;
extracting, by the navigation system, one or more features from the two-dimensional BEV of the spatial points, wherein extracting one or more features from the two-dimensional BEV of the spatial points comprises performing height slicing using a segmented estimate of a ground value for a local area of each anchor point of the plurality anchor points in the two-dimensional BEV and wherein the segmented estimate of the ground value for each anchor point does not use a planar ground representation;
proposing, by the navigation system, one or more regions of the two-dimensional BEV of the spatial points for object detection; and
performing, by the navigation system, object detections on anchor points of the plurality of anchor points in the proposed one or more regions of the two-dimensional BEV of the spatial points.

2. The method of claim 1 , wherein placing the plurality of anchor points in the two-dimensional BEV of the spatial points comprises:
generating the two-dimensional BEV of the spatial points represented in the segmented ground surface representation of objects detected by a LiDAR system based on a grid of coordinates comprising two-dimensional coordinates for each spatial point, the two-dimensional coordinates obtained from a three-dimensional representation of each of one or more objects in physical surroundings detected by the LiDAR system;
sampling the generated two-dimensional BEV for anchors using a predetermined sample size across both axis of the two-dimensional BEV; and
assigning anchor points to the two-dimensional BEV based on the sampling.

3. The method of claim 2 , further comprising, prior to extracting the one or more features from the two-dimensional BEV of the spatial points, pruning one or more of the assigned anchor points from the two-dimensional BEV.

4. The method of claim 1 , wherein proposing one or more regions of the two-dimensional BEV of the spatial points for object detection further comprises:
cropping each of the one or more features extracted from the two-dimensional BEV of the spatial points;
resizing each of the cropped one or more features;
reshaping each of the resized and cropped one or more features;
feeding the reshaped, resized, and cropped one or more features to fully connected layers;
performing Non-Maxima Suppression on anchor bounding boxes for each of the reshaped, resized, and cropped one or more features; and
selecting anchor bounding boxes based on the Non-Maxima Suppression.

5. The method of claim 1 , further comprising using objects detected by performing object detections on anchor points of the plurality of anchor points in the proposed one or more regions of the two-dimensional BEV of the spatial points to learn to predict anchor bounding box coordinates, size, and orientation.

6. The method of claim 1 , further comprising pruning overlapping anchor bounding boxes resulting from performing object detections on anchor points of the plurality of anchor points in the proposed one or more regions of the two-dimensional BEV of the spatial points by performing Non-Maxima Suppression on the detected objects.

7. The method of claim 1 , further comprising determining the segmented estimate of the ground value for the local area of each anchor point by searching a plurality of stored maximum height values for the plurality of anchor points within a predefined rectangular region around each anchor point in the two-dimensional BEV for an anchor point of the plurality of anchor points within the predefined rectangular region having a lowest of the stored maximum height values.

8. A navigation system comprising:
a processor; and
a memory coupled with and readable by the processor and storing therein a set of instructions which, when executed by the processor, causes the processor to perform object detection in a Light Detection And Ranging (LiDAR) point cloud by:
placing a plurality of anchor points in a two-dimensional Bird's Eye View (BEV) of spatial points represented in a segmented ground surface representation of objects detected by a LiDAR system;
extracting one or more features from the two-dimensional BEV of the spatial points, wherein extracting one or more features from the two-dimensional BEV of the spatial points comprises performing height slicing using a segmented estimate of a ground value for a local area of each anchor point of the plurality anchor points in the two-dimensional BEV and wherein the segmented estimate of the ground value for each anchor point does not use a planar ground representation;
proposing one or more regions of the two-dimensional BEV of the spatial points for object detection; and
performing object detections on anchor points of the plurality of anchor points in the proposed one or more regions of the two-dimensional BEV of the spatial points.

9. The navigation system of claim 8 , wherein placing the plurality of anchor points in the two-dimensional BEV of the spatial points comprises:
generating the two-dimensional BEV of the spatial points represented in the segmented ground surface representation of objects detected by a LiDAR system based on a grid of coordinates comprising two-dimensional coordinates for each spatial point, the two-dimensional coordinates obtained from a three-dimensional representation of each of one or more objects in physical surroundings detected by the LiDAR system;
sampling the generated two-dimensional BEV for anchors using a predetermined sample size across both axis of the two-dimensional BEV; and
assigning anchor points to the two-dimensional BEV based on the sampling.

10. The navigation system of claim 9 , wherein the instructions further cause the processor, prior to extracting the one or more features from the two-dimensional BEV of the spatial points, to prune one or more of the assigned anchor points from the two-dimensional BEV.

11. The navigation system of claim 8 , wherein proposing one or more regions of the two-dimensional BEV of the spatial points for object detection further comprises:
cropping each of the one or more features extracted from the two-dimensional BEV of the spatial points;
resizing each of the cropped one or more features;
reshaping each of the resized and cropped one or more features;
feeding the reshaped, resized, and cropped one or more features to fully connected layers;
performing Non-Maxima Suppression on anchor bounding boxes for each of the reshaped, resized, and cropped one or more features; and
selecting anchor bounding boxes based on the Non-Maxima Suppression.

12. The navigation system of claim 8 , wherein the instructions further cause the processor to use objects detected by performing object detections on anchor points of the plurality of anchor points in the proposed one or more regions of the two-dimensional BEV of the spatial points to learn to predict anchor bounding box coordinates, size, and orientation.

13. The navigation system of claim 8 , wherein the instructions further causes the processor to prune overlapping anchor bounding boxes resulting from performing object detections on anchor points of the plurality of anchor points in the proposed one or more regions of the two-dimensional BEV of the spatial points by performing Non-Maxima Suppression on the detected objects.

14. The navigation syste

Assignees

• Nio Technology Anhui Co Ltd

Inventors

• Kumar Arun C S
• Ahuja Disha
• Aithal Ashwath

Classifications

• G01S7/4808 Primary

  Evaluating distance, position or velocity data · CPC title

• G01C21/26

  specially adapted for navigation in a road network · CPC title

• G01S7/4802

  using analysis of echo signal for target characterisation; Target signature; Target cross-section · CPC title

• G01S17/42

  Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S17/46) · CPC title

• G01S17/89

  for mapping or imaging · CPC title