Fault isolation and mitigation upon lane marking misdetection on roadways

Patent metadata

Field	Value
Publication number	US-2023192103-A1
Application number	US-202117557156-A
Country	US
Kind code	A1
Filing date	Dec 21, 2021
Priority date	Dec 21, 2021
Publication date	Jun 22, 2023
Grant date	—

Abstract

A system for a vehicle includes a plurality of sensors onboard the vehicle and a controller. A first sensor of the plurality of sensors is configured to detect lane markings on a roadway. The controller is configured to store data from the plurality of sensors. In response to receiving an indication indicating a misdetection of lane markings on the roadway based on data received from the first sensor, the controller is configured to execute in parallel a plurality of procedures configured to detect a plurality of causes for the misdetection of lane markings, respectively, based on the stored data; isolate one of the causes as a root cause for the misdetection of lane markings; and provide a response for mitigating the misdetection of lane markings on the roadway based on the root cause for the misdetection of lane markings.

First claim

What is claimed is:

1 . A system for a vehicle comprising:
a plurality of sensors onboard the vehicle, wherein a first sensor of the plurality of sensors is configured to detect lane markings on a roadway; and a controller configured to:
store data from the plurality of sensors; and
in response to receiving an indication indicating a misdetection of lane markings on the roadway based on data received from the first sensor:
execute in parallel a plurality of procedures configured to detect a plurality of causes for the misdetection of lane markings, respectively, based on the stored data;
isolate one of the causes as a root cause for the misdetection of lane markings; and
provide a response for mitigating the misdetection of lane markings on the roadway based on the root cause for the misdetection of lane markings.

2 . The system of claim 1 wherein the procedures are configured to detect whether the misdetection of lane markings is caused by any of:
the first sensor being faulty;
absence of lane markings on the roadway;
presence of rain or snow obscuring lane markings on the roadway;
presence of glare or shade obscuring lane markings on the roadway;
presence of an obstacle in a field of view of the first sensor;
presence of construction on the roadway;
a change in lane configuration of the roadway; and
the roadway being unpaved.

3 . The system of claim 1 wherein the response includes following a second vehicle ahead of the vehicle, informing an occupant of the vehicle to take over control of driving the vehicle, switching to a second sensor of the plurality of sensors for lane marking detection, and/or scheduling service.

4 . The system of claim 1 wherein one of the procedures executed by the controller is configured to:
process an image captured by the first sensor;
perform clustering and filtering of pixels in the image;
perform first and second curve fittings on the filtered pixels; and
determine whether a lane marking exists on the roadway based on the second curve fitting.

5 . The system of claim 1 wherein one of the procedures executed by the controller is configured to:
process images captured by two sensors of the plurality of sensors;
determine, for each of the two sensors, number of images with and without lane markings;
calculate, for each of the two sensors, a ratio of the number of images with or without lane markings to a total number of processed images; and
determine whether a lane marking exists on the roadway based on the ratios for the two sensors.

6 . The system of claim 1 wherein one of the procedures executed by the controller is configured to confirm whether the first sensor is faulty by comparing a raw image of the roadway captured by the first sensor to a raw image of the roadway captured by a second sensor of the plurality of sensors.

7 . The system of claim 1 wherein one of the procedures executed by the controller is configured to confirm whether the first sensor is faulty by comparing an objected detected in an image of the roadway captured by the first sensor to an object detected in an image of the roadway captured by a second sensor of the plurality of sensors.

8 . The system of claim 1 wherein one of the procedures executed by the controller is configured to detect whether the roadway is unpaved based on images captured by the first sensor, map information about the roadway, and data from an inertial measurement unit in the vehicle.

9 . The system of claim 1 wherein one of the procedures executed by the controller is configured to detect presence of rain or snow in images captured by the first sensor using a machine learning model.

10 . The system of claim 1 wherein one of the procedures executed by the controller is configured to detect presence of glare or shade in images captured by the first sensor based on intensity of pixels in the images.

11 . The system of claim 1 wherein one of the procedures executed by the controller is configured to detect presence of construction on the roadway based on construction objects detected in images captured by one of the plurality of sensors.

12 . The system of claim 1 wherein one of the procedures executed by the controller is configured to detect a change in lane configuration of the roadway by detecting at least one of a change in number of lanes in the roadway and a change in a relationship between the number of lanes and a lane occupied by the vehicle.

13 . The system of claim 1 wherein one of the procedures executed by the controller is configured to determine, based on images captured by a second sensor of the plurality of sensors and data received from a third sensor of the plurality of sensors, whether a second vehicle is ahead of the vehicle within a predetermined distance from the vehicle and a size of the second vehicle.

14 . The system of claim 13 wherein one of the procedures executed by the controller is configured to determine whether to follow the second vehicle based on the predetermined distance.

15 . The system of claim 13 wherein one of the procedures executed by the controller is configured to determine whether to drive the drive on another trajectory in response to the predetermined distance being greater than a predetermined distance.

16 . A method for a vehicle comprising:
storing data from a plurality of sensors onboard the vehicle; receiving an indication indicating a misdetection of lane markings on a roadway based on data received from a first sensor of the plurality of sensors; in response to receiving the indication, executing in parallel a plurality of procedures configured to detect a plurality of causes for the misdetection of lane markings, respectively, based on the stored data; identifying one of the causes as a root cause for the misdetection of lane markings; and providing a response for mitigating the misdetection of lane markings on the roadway based on the root cause for the misdetection of lane markings, wherein the response includes following a second vehicle ahead of the vehicle, informing an occupant of the vehicle to take over control of driving the vehicle, switching to a second sensor of the plurality of sensors for lane marking detection, and/or scheduling service.

17 . The method of claim 16 further comprising detecting, using the procedures, whether the misdetection of lane markings is caused by any of:
the first sensor being faulty;
absence of lane markings on the roadway;
presence of rain or snow obscuring lane markings on the roadway;
presence of glare or shade obscuring lane markings on the roadway;
presence of an obstacle in a field of view of the first sensor;
presence of construction on the roadway;
a change in lane configuration of the roadway; and
the roadway being unpaved.

18 . The method of claim 16 further comprising:
processing an image captured by the first sensor;
performing clustering and filtering of pixels in the image;
performing first and second curve fittings on the filtered pixels; and
determining whether a lane marking exists on the roadway based on the second curve fitting.

19 . The method of claim 16 further comprising:
processing images captured by two sensors of the plurality of sensors;
determining, for each of the two sensors, number of images with and without lane markings;
calculating, for each of the two sensors, a ratio of the number of images with or without lane markings to a total number of processed images; and
determining whether a lane marking exists on

Assignees

• Gm Global Tech Operations Llc

Inventors

• Lin Wen-Chiao
• Yang Zhongyu
• Hu Yao
• Du Xinyu
• Zhang Yilu

Classifications

• B60W50/02 Primary

  Ensuring safety in case of control system failures, e.g. by diagnosing, circumventing or fixing failures · CPC title

• B60W2050/0072

  Controller asks driver to take over · CPC title

• B60W30/12

  Lane keeping · CPC title

• B60W2050/0215

  Sensor drifts or sensor failures · CPC title

• B60W2555/20

  Ambient conditions, e.g. wind or rain · CPC title