Automated detection of specular reflecting road surfaces using polarimetric image data

Patent metadata

Field	Value
Publication number	US-12291232-B2
Application number	US-202318178748-A
Country	US
Kind code	B2
Filing date	Mar 6, 2023
Priority date	Mar 6, 2023
Publication date	May 6, 2025
Grant date	May 6, 2025

Abstract

A detection system for a host vehicle includes a camera, global positioning system (“GPS”) receiver, compass, and electronic control unit (“ECU”). The camera collects polarimetric image data forming an imaged drive scene inclusive of a road surface illuminated by the Sun. The GPS receiver outputs a present location of the vehicle as a date-and-time-stamped coordinate set. The compass provides a directional heading of the vehicle. The ECU determines the Sun's location relative to the vehicle and camera using an input data set, including the present location and directional heading. The ECU also detects a specular reflecting area or areas on the road surface using the polarimetric image data and Sun's location, with the specular reflecting area(s) forming an output data set. The ECU then executes a control action aboard the host vehicle in response to the output data set.

First claim

What is claimed is:

1. A detection system for a host vehicle, comprising:
a polarimetric camera configured to collect polarized image data, the polarized image data forming an imaged drive scene inclusive of a road surface illuminated by the Sun;
a global positioning system (“GPS”) receiver operable for outputting a present location of the host vehicle as a date and time-stamped coordinate set;
a compass configured to provide a directional heading of the host vehicle; and
an electronic control unit (“ECU”) in communication with the camera, the GPS receiver, and the compass, wherein the ECU is configured to:
determine a location of the Sun relative to the host vehicle and the camera using an input data set, the input data set including the present location of the host vehicle and the directional heading of the host vehicle from the GPS receiver and the compass, respectively;
detect a specular reflecting area on the road surface within the imaged drive scene using the polarized image data and the location of the Sun, wherein the specular reflecting area forms an output data set, and wherein the ECU is configured to detect the specular reflecting area by:
detecting that an image pixel in the polarized image data is not specular if a delta value is small, the delta value being an absolute value of a difference between an estimated angle of linear polarization (AoLP) and a measured AoLP;
selectively applying a light reflecting and scattering model that assumes a dominant light source is located along a straight line extending in front of the host vehicle;
recalculating the delta value as a recalculated delta value using the light reflecting and scattering mode; and
confirming that the image pixel corresponds to the specular reflecting area when the recalculated delta value is close to zero; and
execute a control action aboard the host vehicle in response to the output data set.

2. The system of claim 1 , wherein the ECU is configured to:
measure the measured AoLP using the polarimetric camera;
estimate the estimated AoLP using the location of the Sun and the 3D location of the imaged pixel, assuming that the imaged pixel visualizes the road surface;
calculate the delta value as an absolute value of a difference between the estimated AoLP and the measured AoLP; and
detect the specular reflecting area using the delta value.

3. The system of claim 1 , wherein the host vehicle is a motor vehicle having a vehicle body, and wherein the polarimetric camera is connected to the vehicle body.

4. The system of claim 3 , wherein the polarimetric camera is a color-polarimetric camera.

5. The system of claim 3 , wherein the polarimetric camera is a monochromatic-polarimetric camera.

6. The system of claim 1 , wherein the ECU is in communication with a path planning control module of the host vehicle, and is configured to provide the output data set to the path planning control module as at least part of the control action.

7. The system of claim 1 , wherein the ECU is in communication with a display screen or a heads-up display (“HUD”), and configured to display a graphical representation of the imaged drive scene on the display screen and/or the HUD as at least part of the control action.

8. A method for use with a detection system aboard a host vehicle, comprising:
receiving, via an electronic control unit (“ECU”) of the host vehicle, an input data set from a global positioning system (“GPS”) receiver and a compass of the host vehicle, the input set including a present location of the host vehicle and a directional heading of the host vehicle from the GPS receiver and the compass, respectively;
receiving polarized image data from a polarimetric camera, wherein the polarized image data forms an imaged drive scene inclusive of a road surface illuminated by the Sun, and wherein the present location of the host vehicle includes a date-and-time-stamped coordinate set;
determining a location of the Sun relative to the host vehicle and the polarimetric camera using the input data set;
detecting a specular reflecting area on the road surface within the imaged drive scene using the polarized image data and the location of the Sun, via the ECU, wherein the specular reflecting area forms an output data set, wherein detecting the specular reflecting area includes:
detecting that an image pixel is not specular if a delta value is small, the delta value being an absolute value of a difference between an estimated angle of linear polarization (AoLP) and measured AoLP;
selectively applying a light reflecting and scattering model that assumes a dominant light source is located along a straight line extending in front of the host vehicle;
recalculating the delta value as a recalculated delta value using the light reflecting and scattering mode; and
confirming that the image pixel corresponds to the specular reflecting area when the recalculated delta value is close to zero; and

executing a control action aboard the host vehicle via the ECU in response to the output data set.

9. The method of claim 8 , comprising:
measuring the measured AoLP via the ECU using the polarimetric camera;
estimating the estimated AoLP using the location of the Sun and the 3D location of the imaged pixel assuming that the imaged pixel visualizes the road surface;
calculating a delta value as an absolute value of a difference between the estimated AoLP and the measured AoLP; and
detecting the specular reflecting area via the ECU using the delta value.

10. The method of claim 8 , wherein the host vehicle is a motor vehicle having a vehicle body, the polarimetric camera is a body-mounted camera that is connected to the vehicle body, and receiving the input set includes receiving the polarized image data from the body-mounted camera over a vehicle communication network.

11. The method of claim 10 , wherein receiving the polarimetric image data from the body-mounted camera includes receiving red-green-blue (“RGB”)-polarimetric image data from the body-mounted camera.

12. The method of claim 10 , further comprising:
communicating the output data set to a path planning control module of the host vehicle as at least part of the control action.

13. The method of claim 10 , further comprising:
displaying a graphical representation of the imaged drive scene on a display screen and/or a heads-up display of the host vehicle as at least part of the control action.

14. The method of claim 10 , wherein receiving the polarized image data from the body-mounted camera includes receiving monochromatic-polarimetric image data from the body-mounted camera.

15. A host vehicle, comprising:
a vehicle body;
road wheels connected to the vehicle body; and
a detection system including:
a polarimetric camera configured to collect polarized image data, the polarized image data forming an imaged drive scene inclusive of a road surface illuminated by the Sun;
a global positioning system (“GPS”) receiver operable for outputting a present location of the host vehicle as a date and time-stamped coordinate set;
a compass configured to provide a directional heading of the host vehicle; and
an electronic control unit (“ECU”) in communication with the polarimetric camera, the GPS receiver, and the compass, wherein the ECU is configured to:
determine a location of the Sun relative to the host vehicle and the polarimetric camera using an input data set, the input data set including the present location and the directional heading from the GPS receiver and the compass, respectively;
detect a specular reflecting area on the road surface within the imaged drive scene

Assignees

• Gm Global Tech Operations Llc

Inventors

• Baltaxe Michael
• Philipp Tzvi
• Levi Dan

Classifications

• B60K35/28

  characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver · CPC title

• B60K35/23

  Head-up displays [HUD] (optical aspects of head-up displays G02B27/01) · CPC title

• B60K2360/175

  Autonomous driving · CPC title

• G06V20/588 Primary

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• B60K35/00

  Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles · CPC title