Methods for handling sensor failures in autonomous driving vehicles

Patent metadata

Field	Value
Publication number	US-11427211-B2
Application number	US-201816011526-A
Country	US
Kind code	B2
Filing date	Jun 18, 2018
Priority date	Jun 18, 2018
Publication date	Aug 30, 2022
Grant date	Aug 30, 2022

Abstract

According to some embodiments, described herein is a system and method for handling sensor failures in autonomous driving vehicles (ADV) that is navigating in a world coordination as an absolute coordination system. When the ADV encounters a sensor failure, but still has at least one camera working properly, the sensor failure handling system can switch the ADV from navigating in the world coordination to a local coordination, in which the ADV relies camera-based obstacle detection and lane mark detection to drive safely until human dis-engagement or until the ADV is parked along a road side.

First claim

What is claimed is:

1. A computer-implemented method of operating an autonomous driving vehicle (ADV), the computer-implemented method comprising:
detecting, by a sensor failure handling module, a failure in one or more sensors in the ADV that is navigating in a world coordinate system as an absolute coordinate system;
broadcasting, by the sensor failure handling module, a message indicating the failure to a plurality of autonomous driving modules in the ADV through a message channel, wherein each of the plurality of autonomous driving modules subscribes to the message channel;
in response to receiving the message, switching, by the plurality of subscribing autonomous driving modules, the ADV from navigating in the world coordinate system to navigating in a local coordinate system, which is relative to a current location of the ADV within a lane in which the ADV is driving; and
planning, by the plurality of subscribing autonomous driving modules, a trajectory based on sensor data from one or more cameras mounted on the ADV, to drive the ADV within the lane based on the local coordinate system without using sensor data from a light detection and ranging (LIDAR) sensor or a global positioning system (GPS) sensor, wherein the ADV determines, based on a number of cameras that are still functioning, whether to track the lane and slow down until human disengagement, or track a plurality of lanes and pull up on a side of a road.

2. The computer-implemented method of claim 1 , wherein the plurality of subscribing autonomous driving modules includes a planning module.

3. The computer-implemented method of claim 1 , wherein detecting the failure in the one or more sensors comprises detecting no data received from a sensor, a longer delay in receiving data from the sensor, or a standard deviation in data generated by the sensor, wherein the standard deviation is greater than a predetermined threshold.

4. The computer-implemented method of claim 2 , wherein prior to the detecting of the failure in the one or more sensors the planning module uses sensor data that is combined from the LIDAR sensor, the GPS sensor, and the one or more cameras.

5. The computer-implemented method of claim 1 , wherein when one of the one or more cameras is functioning, the ADV uses sensor data from the camera to detect and track the lane, and slow down until the human disengagement.

6. The computer-implemented method of claim 1 , wherein when two or more cameras of the one or more cameras are functioning, the ADV uses sensor data from the two or more cameras to detect and track the plurality of lanes, and pull up on the side of the road.

7. The computer-implemented method of claim 1 , wherein the sensor failure handling module is configured to monitor a temperature of the ADV, CPU utilization, memory utilization and disk utilization on a computing device in the ADV, and a plurality of sensors in the ADV.

8. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations of operating an autonomous driving vehicle (ADV), the operations comprising:
detecting, by a sensor failure handling module, a failure in one or more sensors in the ADV that is navigating in a world coordinate system as an absolute coordinate system;
broadcasting, by the sensor failure handling module, a message indicating the failure to a plurality of autonomous driving modules in the ADV through a message channel, wherein each of the plurality of autonomous driving modules subscribes to the message channel;
in response to receiving the message, switching, by the plurality of subscribing autonomous driving modules, the ADV from navigating in the world coordinate system to navigating in a local coordinate system, which is relative to a current location of the ADV within a lane in which the ADV is driving; and
planning, by the plurality of subscribing autonomous driving modules, a trajectory based on sensor data from one or more cameras mounted on the ADV, to drive the ADV within the lane based on the local coordinate system without using sensor data from a light detection and ranging (LIDAR) sensor or a global positioning system (GPS) sensor, wherein the ADV determines, based on a number of cameras that are still functioning, whether to track the lane and slow down until human disengagement, or track a plurality of lanes and pull up on a side of road.

9. The non-transitory machine-readable medium of claim 8 , wherein the plurality of subscribing autonomous driving modules includes a planning module.

10. The non-transitory machine-readable medium of claim 8 , wherein detecting the failure in one or more sensors comprises detecting no data received from a sensor, a longer delay in receiving data from the sensor, or a standard deviation in data generated by the sensor, wherein the standard deviation is greater than a predetermined threshold.

11. The non-transitory machine-readable medium of claim 9 , wherein prior to the detecting of the failure in one or more sensors the planning module uses sensor data that is combined from the LIDAR sensor, the GPS sensor, and the one or more cameras.

12. The non-transitory machine-readable medium of claim 8 , wherein when one of the one or more cameras is functioning, the ADV uses sensor data from the camera to detect and track the lane, and slow down until the human disengagement.

13. The non-transitory machine-readable medium of claim 8 , wherein when two or more cameras of the one or more cameras are functioning, ADV uses sensor data from the two or more cameras to detect and track the plurality of lanes, and pull up on the side of the road.

14. The non-transitory machine-readable medium of claim 8 , wherein the sensor failure handling module is configured to monitor a temperature of the ADV, CPU utilization, memory utilization and disk utilization on a computing device in the ADV, and a plurality of sensors in the ADV.

15. A system for handling sensor failures in an autonomous driving vehicle (ADV), comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations comprising:
detecting, by a sensor failure handling module, a failure in one or more sensors in the ADV that is navigating in a world coordinate system as an absolute coordinate system;
broadcasting, by the sensor failure handling module, a message indicating the failure to a plurality of autonomous driving modules in the ADV through a message channel, wherein each of the plurality of autonomous driving modules subscribes to the message channel;
in response to receiving the message, switching, by the plurality of subscribing autonomous driving modules, the ADV from navigating in the world coordinate system to navigating in a local coordinate system, which is relative to a current location of the ADV within a lane in which the ADV is driving; and
planning, by the plurality of subscribing autonomous driving modules, a trajectory based on sensor data from one or more cameras mounted on the ADV, to drive the ADV within the lane based on the local coordinate system without using sensor data from a light detection and ranging (LIDAR) sensor or a global positioning system (GPS) sensor, wherein the ADV determines, based on a number of cameras that are still functioning, whether to track the lane and slow down until human disengagement, or track a plurality of lanes and pull up on a side of road.

16. The system of claim 15 , wherein the plurality of subscribing autonomous

Assignees

• Baidu Usa Llc

Inventors

• Hu Jiangtao
• Jiang Yifei
• Li Dong
• Zhang Liangliang
• Tao Jiaming
• Luo Qi
• Xiao Xiangquan

Classifications

• G06V20/588

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• B60W50/029 Primary

  Adapting to failures or work around with other constraints, e.g. circumvention by avoiding use of failed parts · CPC title

• B60W2050/0292

  Fail-safe or redundant systems, e.g. limp-home or backup systems · CPC title

• B60W50/0205 Primary

  Diagnosing or detecting failures; Failure detection models · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title