Vehicle sensor occlusion detection

Patent metadata

Field	Value
Publication number	US-12162497-B2
Application number	US-202117517855-A
Country	US
Kind code	B2
Filing date	Nov 3, 2021
Priority date	Nov 3, 2021
Publication date	Dec 10, 2024
Grant date	Dec 10, 2024

Abstract

A system for detecting a road surface includes a processor programmed to identify, in a vehicle sensor field of view, environment features including a sky, a road surface, a road shoulder, based on map data and data received from the one or more vehicle sensors, upon determining a low confidence in identifying an area of the field of view to be a road surface or sky, to receive a polarimetric image, and to determine, based on the received polarimetric image, whether the identified area is a mirage phenomenon, a wet road surface, or the sky. The low confidence is determined upon determining that a road surface has a vanishing edge, a road lane marker is missing, or an anomalous object is present.

First claim

What is claimed is:

1. A system for detecting a road surface in a vehicle, comprising a processor and a memory, the memory storing instructions executable by the processor to:
identify, from a non-polarimetric image including a vehicle sensor field of view, environment features including a sky, a road surface, and/or a road shoulder, based on map data and data received from the one or more vehicle sensors;
receive a polarimetric image based on determining a confidence, that measures a likelihood that one of the environment features in an area of the field of view in the non-polarimetric image is correctly identified, is below a confidence threshold and is therefore a low confidence;
determine, based on the received polarimetric image, whether the identified area is a mirage phenomenon, a wet road surface, or the sky; and
based on the low confidence that the one of the environment features in the non-polarimetric image is correctly identified, operate the vehicle based on the polarimetric image in addition to the non-polarimetric image;
wherein the low confidence is determined upon determining that a road surface has a vanishing edge, a road lane marker is missing, or an anomalous object is present.

2. The system of claim 1 , wherein the instructions further include instructions to determine the mirage phenomenon based on environmental conditions including a temperature and an atmospheric condition.

3. The system of claim 1 , wherein the instructions further include instructions to identify the environment features using semantic segmentation of the non-polarimetric image.

4. The system of claim 1 , wherein the anomalous object includes at least one of a mirrored vehicle or an object with a distorted shape.

5. The system of claim 1 , wherein the instructions further include instructions to determine the mirage phenomenon upon determining (i) the area of the field of view, based on semantic segmentation, to be a sky portion, (ii) the area, based on image features, to be a road surface, (iii) the area, based on localization based on map data and vehicle sensor data, to be a road surface, (iv) the area, based on the received polarimetric image, to be a sky portion, and (v) a probability of atmospheric condition for mirage phenomenon that exceeds a probability threshold.

6. The system of claim 1 , wherein the instructions further include instructions to determine an occluded volume in the field of view upon detecting the mirage phenomenon, and actuate a vehicle actuator based on the determined occluded volume.

7. The system of claim 1 , wherein the instructions further include instructions to determine, based on the received polarimetric image, at least one of: (i) a road surface upon detecting a low to moderate degree of polarization and a direction of polarization matching a road surface polarization direction, (ii) a sky portion upon detecting a low degree of polarization and a direction of polarization matching a skydome prediction of sky, or (iii) a water area upon detecting a high degree of polarization and a direction of polarization matching water direction of polarization.

8. The system of claim 1 , wherein the instructions further include instructions to determine a water area upon detecting a high degree of polarization and a direction of polarization matching water direction of polarization, and actuate a vehicle actuator based on the determined water area.

9. The system of claim 8 , wherein the instructions further include instructions, upon determining the water area on a vehicle path, to actuate the vehicle actuator based on a distance of the vehicle to the water area.

10. A method, comprising:
identifying, from a non-polarimetric image including a vehicle sensor field of view, environment features including a sky, a road surface, and/or a road shoulder, based on map data and data received from the one or more vehicle sensors;
receiving a polarimetric image based on determining a confidence, that measures a likelihood that one of the environment features in an area of the field of view in the non-polarimetric image is correctly identified, is below a confidence threshold and is therefore a low confidence;
determining, based on the received polarimetric image, whether the identified area is a mirage phenomenon, a wet road surface, or the sky; and
based on the low confidence that the one of the environment features in the non-polarimetric image is correctly identified, operating the vehicle based on the polarimetric image in addition to the non-polarimetric image;
wherein the low confidence is determined upon determining that a road surface has a vanishing edge, a road lane marker is missing, or an anomalous object is present.

11. The method of claim 10 , further comprising determining the mirage phenomenon based on environmental conditions including a temperature and an atmospheric condition.

12. The method of claim 10 , further comprising identifying the environment features using semantic segmentation of the non-polarimetric image.

13. The method of claim 10 , wherein the anomalous object includes at least one of a mirrored vehicle or an object with a distorted shape.

14. The method of claim 10 , further comprising determining the mirage phenomenon upon determining (i) the area of the field of view, based on semantic segmentation, to be a sky portion, (ii) the area, based on image features, to be a road surface, (iii) the area, based on localization based on map data and vehicle sensor data, to be a road surface, (iv) the area, based on the received polarimetric image, to be a sky portion, and (v) a probability of atmospheric condition for mirage phenomenon that exceeds a probability threshold.

15. The method of claim 10 , further comprising determining an occluded volume in the field of view upon detecting the mirage phenomenon, and actuate a vehicle actuator based on the determined occluded volume.

16. The method of claim 15 , further comprising determining, based on the received polarimetric image, at least one of: (i) a road surface upon detecting a low to moderate degree of polarization and a direction of polarization matching a road surface polarization direction, (ii) a sky portion upon detecting a low degree of polarization and a direction of polarization matching a skydome prediction of sky, or (iii) a water area upon detecting a high degree of polarization and a direction of polarization matching water direction of polarization.

17. The method of claim 10 , further comprising determining a water area upon detecting a high degree of polarization and a direction of polarization matching water direction of polarization, and actuating a vehicle actuator based on the determined water area.

18. The method of claim 17 , further comprising, upon determining the water area on a vehicle path, actuating the vehicle actuator based on a distance of the vehicle to the water area.

19. The system of claim 1 , wherein the instructions further include instructions to determine an occluded volume based on the detected mirage phenomenon and operate the vehicle based on the determined occluded volume.

20. The system of claim 1 , wherein the instructions further include instructions to, upon determining that no mirage condition is detected, operate the vehicle based on one of the wet road surface or the sky.

Assignees

• Ford Global Tech Llc

Inventors

• Herman david michael
• Lesky Aaron
• Rajvanshi Akshat
• Kettlewell Brian Quinn
• Horowitz Geoffrey

Classifications

• B60W2420/408

  Radar; Laser, e.g. lidar · CPC title

• B60W2420/403

  Image sensing, e.g. optical camera · CPC title

• G06V30/274

  Syntactic or semantic context, e.g. balancing · CPC title

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06V10/143

  Sensing or illuminating at different wavelengths · CPC title