Methods and systems for lidar point cloud anomalies

Patent metadata

Field	Value
Publication number	US-10401866-B2
Application number	US-201715585891-A
Country	US
Kind code	B2
Filing date	May 3, 2017
Priority date	May 3, 2017
Publication date	Sep 3, 2019
Grant date	Sep 3, 2019

Abstract

Systems and method are provided for controlling an autonomous vehicle. In one embodiment, a method for controlling an autonomous vehicle comprises obtaining lidar data from one or more lidar sensors disposed on the autonomous vehicle during operation of the autonomous vehicle, generating a lidar point cloud using the lidar data, making an initial determination, via a processor onboard the autonomous vehicle, of a possible lidar point cloud anomaly based on a comparison of the lidar point cloud with prior lidar point cloud information stored in memory, receiving a notification from a remote module as to whether the possible lidar point cloud anomaly is a confirmed lidar point cloud anomaly, and taking one or more vehicle actions when there is a confirmed lidar point cloud anomaly.

First claim

What is claimed is:

1. A method for controlling an autonomous vehicle, the method comprising:
obtaining lidar data from one or more lidar sensors disposed on the autonomous vehicle during operation of the autonomous vehicle;
generating a lidar point cloud using the lidar data;
making an initial determination, via a processor onboard the autonomous vehicle, using a convolutional neural network model, of a possible lidar point cloud anomaly based on a comparison of the lidar point cloud with prior lidar point cloud information stored in a memory, wherein the autonomous vehicle is determined to have a potential lidar point cloud anomaly when the lidar point cloud is determined to include patterns in common with a known lidar point cloud history from the prior lidar point cloud information for a circumstance that warrants a change in path for the autonomous vehicle;
receiving a notification from a remote module as to whether the possible lidar point cloud anomaly is a confirmed lidar point cloud anomaly, the confirmed lidar point cloud anomaly comprising a confirmation that a change in path for the autonomous vehicle is warranted; and
taking one or more vehicle actions when there is a confirmed lidar point cloud anomaly.

2. The method of claim 1 , wherein the possible lidar point cloud anomaly is confirmed manually by a remote human operator associated with the remote module.

3. The method of claim 1 , wherein the step of making the initial determination comprises making the initial determination, via the processor onboard the autonomous vehicle, based on a determination as to whether one or more patterns of the lidar point cloud are consistent with and shared by one or more known lidar point cloud histories from the prior lidar point cloud information for one or more circumstances that include one or more of the following: a construction site, an emergency scene, an emergency vehicle, a traffic impediment, or a road blockage.

4. The method of claim 3 , wherein the step of making the initial determination comprises making the initial determination, via the processor onboard the autonomous vehicle, based on a determination as to whether one or more patterns of the lidar point cloud are consistent with and shared by one or more known lidar point cloud histories from the prior lidar point cloud information for a circumstance that includes a construction site.

5. The method of claim 3 , wherein the step of making the initial determination comprises making the initial determination, via the processor onboard the autonomous vehicle, based on a determination as to whether one or more patterns of the lidar point cloud are consistent with and shared by one or more known lidar point cloud histories from the prior lidar point cloud information for a circumstance that includes an emergency scene with an emergency vehicle.

6. The method of claim 3 , wherein the step of making the initial determination comprises making the initial determination, via the processor onboard the autonomous vehicle, based on a determination as to whether one or more patterns of the lidar point cloud are consistent with and shared by one or more known lidar point cloud histories from the prior lidar point cloud information for a circumstance that includes a traffic impediment.

7. The method of claim 3 , wherein the step of making the initial determination comprises making the initial determination, via the processor onboard the autonomous vehicle, based on a determination as to whether one or more patterns of the lidar point cloud are consistent with and shared by one or more known lidar point cloud histories from the prior lidar point cloud information for a circumstance that includes a road blockage.

8. The method of claim 1 , wherein the step of transmitting information comprises:
transmitting an image of the lidar point cloud, along with video footage of a roadway surrounding the autonomous vehicle, to the remote module.

9. The method of claim 8 , wherein the lidar point cloud anomaly is confirmed manually by a human remote operator associated with the remote module based at least in part on the image of the lidar point cloud and the video footage.

10. The method of claim 1 , wherein the step of taking one or more vehicle actions comprises taking an alternate route for the autonomous vehicle based on the confirmed lidar point cloud anomaly.

11. The method of claim 10 , wherein the step of taking an alternate route comprises taking the alternate route along a roadway that is not on a list of blacklisted roadways based on reported lidar point cloud anomalies.

12. The method of claim 1 , wherein the step of taking one or more vehicle actions comprises reporting a current roadway on which the autonomous vehicle is travelling for inclusion on a blacklist for path planning for the autonomous vehicle, one or more other autonomous vehicles, or both.

13. A system for controlling an autonomous vehicle, the system comprising:
an autonomous vehicle computer module configured to at least facilitate, via a processor onboard the vehicle:
obtaining lidar data from one or more lidar sensors disposed on the autonomous vehicle during operation of the autonomous vehicle;
generating a lidar point cloud using the lidar data;
making an initial determination, via a processor onboard the autonomous vehicle, using a convolutional neural network model, of a possible lidar point cloud anomaly based on a comparison of the lidar point cloud with prior lidar point cloud information stored in memory, wherein the autonomous vehicle is determined to have a potential lidar point cloud anomaly when the lidar point cloud is determined to include patterns in common with a known lidar point cloud history from the prior lidar point cloud information for a circumstance that warrants a change in path for the autonomous vehicle; and

a remote computer module disposed remote from the autonomous vehicle and configured to at least facilitate confirming the possible lidar point cloud anomaly, the confirmed lidar point cloud anomaly comprising a confirmation that a change in path for the autonomous vehicle is warranted;
wherein the autonomous vehicle computer module is further configured to at least facilitate:
receiving a notification from the remote computer module as to whether the possible lidar point cloud anomaly is a confirmed lidar point cloud anomaly, the confirmed lidar point cloud anomaly comprising a confirmation that a change in path for the autonomous vehicle is warranted; and
taking one or more vehicle actions when there is a confirmed lidar point cloud anomaly.

14. The system of claim 13 , wherein the remote computer module is configured to at least facilitate receiving manual input from a remote human operator that manually confirms the possible lidar point cloud anomaly.

15. The system of claim 13 , wherein the autonomous vehicle computer module is configured to at least facilitate taking an alternate route for the autonomous vehicle based on reported lidar point cloud anomalies based on the confirmed lidar point cloud anomaly, and wherein the autonomous vehicle computer module is configured to at least facilitate reporting a current roadway on which the autonomous vehicle is travelling for inclusion on a blacklist for path planning for the autonomous vehicle, one or more other autonomous vehicles, or both.

16. The system of claim 13 , wherein the autonomous vehicle computer module is configured to at least facilitate, via the processor onboard the vehicle, making the initial determination based on whether one or more patterns of the lidar point cloud are consi

Assignees

• Gm Global Tech Operations Llc

Inventors

• Rust Ian

Classifications

• G01S17/931 Primary

  of land vehicles · CPC title

• G01S17/89 Primary

  for mapping or imaging · CPC title

• G01S7/003

  Transmission of data between radar, sonar or lidar systems and remote stations · CPC title

• G05D1/0238 Primary

  using obstacle or wall sensors (G05D1/0246 and G05D1/0289 take precedence; lidar systems designed for anti-collision purposes G01S17/93) · CPC title

• G05D1/0276

  using signals provided by a source external to the vehicle (involving a plurality of vehicles G05D1/0287; automatically controlling vehicle speed responsive to externally generated signals B60K31/0058) · CPC title