Augmented-reality map assistance to improve driver visibility

Patent metadata

Field	Value
Publication number	US-11975734-B2
Application number	US-202217822004-A
Country	US
Kind code	B2
Filing date	Aug 24, 2022
Priority date	Aug 24, 2022
Publication date	May 7, 2024
Grant date	May 7, 2024

Abstract

A method includes monitoring environmental conditions surrounding a moving vehicle, using an on-board sensor of the moving vehicle, detecting that at least one of the environmental conditions surrounding the moving vehicle is likely to impair visibility for a driver of the moving vehicle, based on an output of the on-board sensor, acquiring static map data for a location surrounding the moving vehicle, acquiring real time sensor data for the location surrounding the moving vehicle, where the real time sensor data comprises outputs of a plurality of on-board sensors of a plurality of other vehicles, generating an augmented reality view of the location surrounding the moving vehicle, using the static map data, the real time sensor data, and the output of the on-board sensor, and displaying the augmented reality view of the location surrounding the moving vehicle on a display within the moving vehicle.

First claim

What is claimed is:

1. A method comprising:
monitoring, by a processing system including at least one processor, environmental conditions surrounding a moving vehicle, using an on-board sensor of the moving vehicle;
detecting, by the processing system, that at least one of the environmental conditions surrounding the moving vehicle is likely to impair visibility for a driver of the moving vehicle, based on an output of the on-board sensor;
acquiring, by the processing system, static map data for a location surrounding the moving vehicle;
acquiring, by the processing system, real time sensor data for the location surrounding the moving vehicle, where the real time sensor data comprises outputs of a plurality of on-board sensors of a plurality of other vehicles;
generating, by the processing system, an augmented reality view of the location surrounding the moving vehicle, using the static map data, the real time sensor data, and the output of the on-board sensor; and
displaying, by the processing system, the augmented reality view of the location surrounding the moving vehicle on a display within the moving vehicle.

2. The method of claim 1 , wherein the on-board sensor comprises at least one of: a sonar system, a lidar system, a radar system, a camera system, a global positioning system, a moisture sensor, a proximity sensor, a light sensor, a barometer, a thermometer, or an anemometer.

3. The method of claim 1 , wherein the detecting is in response to a detection of a predefined condition in the output of the on-board sensor.

4. The method of claim 1 , wherein the detecting is in response to receiving a signal from a user of the moving vehicle.

5. The method of claim 1 , wherein the detecting is in response to receiving an alert from a remote server.

6. The method of claim 1 , wherein the static map data comprises historical street level views or images of the location surrounding the moving vehicle.

7. The method of claim 1 , wherein the real time sensor data for the location surrounding the moving vehicle comprises at least one of: data indicative of weather conditions or data indicative of traffic conditions.

8. The method of claim 1 , wherein the real time sensor data is acquired directly from the plurality of other vehicles.

9. The method of claim 1 , wherein real time sensor data is acquired from a remote server in communication with a database that receives the real time sensor data from the plurality of other vehicles.

10. The method of claim 1 , wherein the augmented reality view of the location surrounding the moving vehicle comprises a view of the location surrounding the moving vehicle that is free of the at least one of the environmental conditions that was determined to be likely to impair visibility for the driver of the moving vehicle.

11. The method of claim 1 , wherein the augmented reality view of the location surrounding the moving vehicle comprises a video image of a road ahead of the moving vehicle in which the at least one of the environmental conditions is digitally removed.

12. The method of claim 1 , wherein the display comprises at least one of: a heads up display, a dashboard display, or an augmented reality-enabled windshield.

13. The method of claim 1 , further comprising:
sending, by the processing system, the output of the on-board sensor to a remote database that stores outputs of a plurality of on-board sensors of a plurality of vehicles.

14. The method of claim 13 , wherein each output of the outputs of the plurality of on-board sensors is associated with metadata that indicates a location at which the each output was recorded.

15. The method of claim 13 , wherein each output of the outputs of the plurality of on-board sensors is associated with metadata that indicates a time at which the each output was recorded.

16. The method of claim 15 , wherein the each output is automatically deleted from the remote database when an age of the each output, as indicated by the time at which the each output was recorded, is greater than a predefined threshold.

17. The method of claim 13 , wherein the plurality of on-board sensors of the plurality of vehicles include at least one of: a sonar system, a lidar system, a radar system, a camera system, a global positioning system, a moisture sensor, a proximity sensor, a light sensor, a barometer, a thermometer, or an anemometer.

18. The method of claim 1 , further comprising:
repeating the monitoring, the detecting the acquiring the static map data, the acquiring the real time sensor data, the generating, and the displaying until an occurrence of a predefined event is detected.

19. A non-transitory computer-readable storage medium storing instructions which, when executed by a processing system including at least one processor, cause the processing system to perform operations, the operations comprising:
monitoring environmental conditions surrounding a moving vehicle, using an on-board sensor of the moving vehicle;
detecting that at least one of the environmental conditions surrounding the moving vehicle is likely to impair visibility for a driver of the moving vehicle, based on an output of the on-board sensor;
acquiring static map data for a location surrounding the moving vehicle;
acquiring real time sensor data for the location surrounding the moving vehicle, where the real time sensor data comprises outputs of a plurality of on-board sensors of a plurality of other vehicles;
generating an augmented reality view of the location surrounding the moving vehicle, using the static map data, the real time sensor data, and the output of the on-board sensor; and
displaying the augmented reality view of the environment surrounding the moving vehicle on a display within the moving vehicle.

20. A system comprising:
a processing system including at least one processor; and
a non-transitory computer-readable medium storing instructions which, when executed by the processing system, cause the processing system to perform operations, the operations comprising:
monitoring environmental conditions surrounding a moving vehicle, using an on-board sensor of the moving vehicle;
detecting that at least one of the environmental conditions surrounding the moving vehicle is likely to impair visibility for a driver of the moving vehicle, based on an output of the on-board sensor;
acquiring static map data for a location surrounding the moving vehicle;
acquiring real time sensor data for the location surrounding the moving vehicle, where the real time sensor data comprises outputs of a plurality of on-board sensors of a plurality of other vehicles;
generating an augmented reality view of the location surrounding the moving vehicle, using the static map data, the real time sensor data, and the output of the on-board sensor; and
displaying the augmented reality view of the location surrounding the moving vehicle on a display within the moving vehicle.

Assignees

• At & T Ip I Lp

Inventors

• Alam Iftekhar
• Khallil Wasib
• Chang Jonathan
• Patel Bhumit

Classifications

• B60K35/40

  Instruments specially adapted for improving the visibility thereof to the user, e.g. fogging prevention or anti-reflection arrangements · CPC title

• B60K35/23

  Head-up displays [HUD] (optical aspects of head-up displays G02B27/01) · CPC title

• B60K35/81

  for controlling displays · CPC title

• B60W50/14 Primary

  Means for informing the driver, warning the driver or prompting a driver intervention · CPC title

• B60K35/28 Primary

  characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver · CPC title