Autonomous vehicle controlled based upon a lidar data segmentation system

Patent metadata

Field	Value
Publication number	US-2021223402-A1
Application number	US-202117226123-A
Country	US
Kind code	A1
Filing date	Apr 9, 2021
Priority date	Aug 3, 2018
Publication date	Jul 22, 2021
Grant date	—

Abstract

An autonomous vehicle is described herein. The autonomous vehicle includes a lidar sensor system. The autonomous vehicle additionally includes a computing system that executes a lidar segmentation system, wherein the lidar segmentation system is configured to identify objects that are in proximity to the autonomous vehicle based upon output of the lidar sensor system. The computing system further includes a deep neural network (DNN), where the lidar segmentation system identifies the objects in proximity to the autonomous vehicle based upon output of the DNN.

First claim

What is claimed is:

1 . An autonomous vehicle (AV) comprising:
an engine; a braking system; a steering system; a lidar sensor; and a computing system that is in communication with the engine, the braking system, the steering system, and the lidar sensor, wherein the computing system comprises:
a processor; and
memory that stores instructions that, when executed by the processor, cause the processor to perform acts comprising:
receiving lidar data, the lidar data based upon output of the lidar sensor, the lidar data comprising a plurality of points representative of positions of objects in a driving environment of the AV;
assigning a label to a first point in the points that indicates that the first point is representative of ground cover or vegetation based upon output of a deep neural network (DNN) that is configured to classify points as being representative of ground cover or vegetation;
generating a segmentation of the lidar data based upon the label being assigned to the first point, wherein the segmentation is indicative of a second point in the lidar data and a third point in the lidar data being representative of a same object; and
controlling at least one of the engine, the braking system, or the steering system during operation of the AV in the driving environment based upon the segmentation.

2 . The AV of claim 1 , wherein the output of the DNN comprises a probability that the first point is representative of vegetation.

3 . The AV of claim 2 , wherein the label indicates that the first point is representative of vegetation, wherein assigning the label is based upon the probability exceeding a threshold value.

4 . The AV of claim 1 , wherein responsive to receipt of input features pertaining to the first point, the output of the DNN comprises:
a first probability that the first point is representative of vegetation; a second probability that the first point is representative of ground cover; and a third probability that the first point is representative of an object of a type other than vegetation or ground cover.

5 . The AV of claim 1 , wherein the output of the DNN comprises a probability that the first point is representative of ground cover.

6 . The AV of claim 1 , wherein generating the segmentation of the lidar data comprises assigning group labels to the points in the lidar data, each group label indicating one of a plurality of groups of points, each group of points representative of a different respective object in the driving environment.

7 . The AV of claim 6 , wherein generating the segmentation comprises assigning a same first group label to the first point and a fourth point based upon the first point and the fourth point being labeled as representative of vegetation, the first group label indicative of first group that is representative of a vegetation object in the driving environment.

8 . The AV of claim 6 , wherein generating the segmentation comprises assigning different group labels to the first point and a fourth point based upon the first point being labeled as representative of vegetation and the fourth point not being labeled as representative of vegetation.

9 . The AV of claim 1 , wherein generating the segmentation comprises excluding the first point from consideration by a segmentation algorithm based upon the label being assigned to the first point.

10 . The AV of claim 1 , the acts further comprising assigning a respective label to each of a first group of points in the points based upon output of the DNN, the labels assigned to the first group of points indicating that the first group of points are representative of vegetation or ground cover in the driving environment, wherein generating the segmentation is based further upon the labels assigned to the first group of points.

11 . The AV of claim 10 , wherein generating the segmentation comprises excluding the first point and the first group of points from consideration by a segmentation algorithm based upon the labels being assigned to the first point and the first group of points.

12 . A method for controlling operation of an autonomous vehicle (AV), comprising:
receiving lidar data from a lidar sensor system of the AV, the lidar data based upon output of at least one lidar sensor, the lidar data comprising a plurality of points representative of positions of objects in a driving environment of the AV; assigning a label to a first point in the points based upon output of a deep neural network (DNN) that is configured to output a probability that a point in lidar data is representative of ground cover or vegetation, the label indicating that the first point is representative of ground cover or vegetation in the driving environment; generating a segmentation of the lidar data based upon the label being assigned to the first point, wherein the segmentation is indicative of a second point in the lidar data and a third point in the lidar data being representative of a same object; and controlling, based upon the segmentation, at least one of an engine of the AV, a braking system of the AV, or a steering system of the AV during operation of the AV in the driving environment.

13 . The method of claim 12 , wherein the output of the DNN comprises a probability that the first point is representative of vegetation.

14 . The method of claim 12 , wherein the output of the DNN comprises a probability that the first point is representative of ground cover.

15 . The method of claim 14 , wherein the label indicates that the first point is representative of ground cover, wherein assigning the label is based upon the probability exceeding a threshold value.

16 . The method of claim 12 , wherein generating the segmentation of the lidar data comprises assigning group labels to the points in the lidar data, each group label indicating one of a plurality of groups of points, each group of points representative of a different respective object in the driving environment.

17 . The method of claim 12 , wherein generating the segmentation comprises executing a lidar segmentation algorithm over the lidar data based upon the label being assigned to the first point.

18 . The method of claim 12 , the acts further comprising assigning a respective label to each of a first group of points in the points based upon output of the DNN, the labels assigned to the first group of points indicating that the first group of points are representative of vegetation or ground cover in the driving environment, wherein generating the segmentation is based further upon the labels assigned to the first group of points.

19 . The method of claim 18 , wherein generating the segmentation comprises excluding the first point and the first group of points from consideration by a segmentation algorithm based upon the labels being assigned to the first point and the first group of points.

20 . An autonomous vehicle (AV) comprising:
a computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:
receiving a lidar point cloud from a lidar sensor system of the AV, the lidar point cloud based upon output of at least one lidar sensor, the lidar point cloud comprising a plurality of points representative of positions of objects in a driving environment of the AV;
assigning a label to a first point in the points that indicates that the first point is representative of vegetation in the driving environment based upon output of a deep neural network (DNN), wh

Assignees

• Gm Global Tech Operations Llc

Inventors

• Allais Andrea
• Cadien Adam Samuel
• Branson Elliot
• Xie William Gongshu
• Chambers Micah Christopher

Classifications

• G01S17/931 Primary

  of land vehicles · CPC title

• G01S7/4808 Primary

  Evaluating distance, position or velocity data · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title