Autonomous vehicle controlled based upon a lidar data segmentation system

Patent metadata

Field	Value
Publication number	US-11853061-B2
Application number	US-202117491453-A
Country	US
Kind code	B2
Filing date	Sep 30, 2021
Priority date	Aug 3, 2018
Publication date	Dec 26, 2023
Grant date	Dec 26, 2023

Abstract

An autonomous vehicle is described herein. The autonomous vehicle includes a lidar sensor system. The autonomous vehicle additionally includes a computing system that executes a lidar segmentation system, wherein the lidar segmentation system is configured to identify objects that are in proximity to the autonomous vehicle based upon output of the lidar sensor system. The computing system further includes a deep neural network (DNN), where the lidar segmentation system identifies the objects in proximity to the autonomous vehicle based upon output of the DNN.

First claim

What is claimed is:

1. An autonomous vehicle (AV) comprising:
a lidar sensor; and
a computing system that is in communication with the lidar sensor, wherein the computing system comprises:
a processor; and
memory that stores instructions that, when executed by the processor, cause the processor to perform acts comprising:
receiving lidar data, the lidar data based upon output of the lidar sensor, the lidar data comprising a plurality of points representative of positions of objects in a driving environment of the AV;
providing a first input feature that pertains to a first point in the lidar data to a deep neural network (DNN), wherein the first input feature is a distance from the first point to a next-closest point in the plurality of points, wherein the DNN generates first output that pertains to the first point responsive to receiving the first input feature;
providing a second input feature that pertains to a second point in the lidar data to the DNN, wherein the DNN generates second output that pertains to the second point responsive to receiving the second input feature; and
assigning respective labels to the first point and the second point based upon the first output and the second output of the DNN, wherein the labels indicate that the first point and the second point are representative of a same object in the driving environment.

2. The AV of claim 1 , wherein the first output of the DNN comprises a first vector of features pertaining to the first point, and the second output of the DNN comprises a second vector of features pertaining to the second point.

3. The AV of claim 2 , wherein the assigning the respective labels is based further upon a distance between the first vector and the second vector.

4. The AV of claim 2 , wherein the assigning the respective labels is based upon a greatest difference between corresponding features of the first vector and the second vector.

5. The AV of claim 2 , wherein the labels assigned to the first point and the second point indicate that the first point and the second point belong to a group of points in the plurality of points that are representative of the same object in the driving environment, wherein each of the points in the group of points has a vector of features pertaining thereto, the vectors of features being output by the DNN.

6. The AV of claim 5 , wherein each of the vectors of features comprises a respective first feature value, the acts further comprising:
responsive to determining that the first feature value of the first vector of features is greater than a threshold number of standard deviations away from a mean of the first feature values of the vectors of features, updating the label assigned to the first point to indicate that the first point does not belong to the group of points.

7. The AV of claim 1 , wherein a plurality of first input features that pertain to the first point in the lidar data are provided to the DNN, wherein the DNN generates the first output that pertains to the first point responsive to receiving the plurality of first input features, and wherein the plurality of first input features comprises the first input feature and a number of points in the plurality of points that are within a threshold distance of the first point.

8. The AV of claim 1 , wherein a plurality of first input features that pertain to the first point in the lidar data are provided to the DNN, wherein the DNN generates the first output that pertains to the first point responsive to receiving the plurality of first input features, and wherein the plurality of first input features comprises the first input feature and a distance from the first point to the AV.

9. The AV of claim 1 , wherein the labels assigned to the first point and the second point indicate that the first point and the second point belong to a group of points in the plurality of points that are representative of the same object in the driving environment.

10. The AV of claim 9 , the acts further comprising:
responsive to determining that a distance between the first point and a next-closest point in the group of points is greater than a threshold distance, updating the label assigned to the first point to indicate that the first point does not belong to the group of points.

11. The AV of claim 9 , the acts further comprising:
updating the label assigned to the first point to indicate that the first point does not belong to the group of points based upon an angle formed by the first point and two other points of the group of points.

12. The AV of claim 11 , wherein the two other points of the group of points are points adjacent to the first point along a scan line of points in the lidar data.

13. The AV of claim 1 , further comprising:
a braking system;
a steering system; and
a motor, the acts further comprising controlling at least one of the braking system, the steering system or the motor to effectuate motion of the AV based upon the labels being assigned to the first point and the second point.

14. A method comprising:
generating lidar data by way of a lidar sensor mounted on an autonomous vehicle (AV), the lidar data comprising a plurality of points representative of positions of objects in a driving environment of the AV;
providing a first input feature that pertains to a first point in the lidar data to a deep neural network (DNN), wherein the DNN generates first output that pertains to the first point responsive to receiving the first input feature;
providing a second input feature that pertains to a second point in the lidar data to the DNN, wherein the DNN generates second output that pertains to the second point responsive to receiving the second input feature;
assigning respective labels to the first point and the second point based upon the first output and the second output of the DNN, wherein the labels indicate that the first point and the second point are representative of a same object in the driving environment, and wherein the labels indicate that the first point and the second point belong to a group of points in the plurality of points that are representative of the same object in the driving environment, and
updating the label assigned to the first point to indicate that the first point and the second point do not belong to the same object based upon an angle formed by the first point, the second point, and a third point in the group of points.

15. The method of claim 14 , wherein the lidar data comprises a plurality of scan lines, wherein the first point and the second point lie along a same scan line in the scan lines.

16. The method of claim 15 , wherein the third point lies along the same scan line as the first point and the second point.

17. A computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:
receiving a lidar point cloud comprising a plurality of points representative of positions of objects in a driving environment of an AV, the lidar point cloud based upon output of a lidar sensor mounted on the AV;
providing a first input feature that pertains to a first point in the lidar data to a deep neural network (DNN), wherein the first input feature is a number of points in the plurality of points that are within a threshold distance of the first point, wherein the DNN outputs first data that pertains to the first point responsive to receiving the first input feature;
providing a second input feature that pertains to a second point in the lidar data to the DNN, wherein the DNN generates outputs second data that pertain

Assignees

• Gm Global Tech Operations Llc

Inventors

• Allais Andrea
• Chambers Micah Christopher
• Xie William Gongshu
• Cadien Adam Samuel
• Branson Elliot

Classifications

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G05D1/0088 Primary

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title

• B60W10/06

  including control of combustion engines · CPC title

• B60W10/18

  including control of braking systems · CPC title