Enhanced augmented reality vehicle pathway

Patent metadata

Field	Value
Publication number	US-11482191-B2
Application number	US-202017065714-A
Country	US
Kind code	B2
Filing date	Oct 8, 2020
Priority date	Oct 8, 2020
Publication date	Oct 25, 2022
Grant date	Oct 25, 2022

Abstract

A User-centric Enhanced Pathway (UEP) system may provide vehicle pathway guidance while driving using an augmented reality display system in the event of sun glare. The system uses a route observer module programmed to predict sun glare, by obtaining real-time information about the interior and exterior vehicle environment, heading, speed, and other data. The route observer module may also use an inward-facing camera to determine when the driver is squinting, which may increase the probability function that predicts when the driver is experiencing sun glare. When sun glare is predicted, the route observer sends the weighted prediction function and input signals to a decision module that uses vehicle speed, location, and user activity to determine appropriate guidance output for an enhanced pathway using an Augmented Reality (AR) module. The AR module may adjust brightness, contrast, and color of the AR information based on observed solar glare.

First claim

That which is claimed is:

1. A method for providing augmented reality (AR) vehicle guidance for solar glare mitigation in a vehicle, comprising:
receiving, via a camera sensor, an image frame of a user driving the vehicle;
determining, based on the image frame, an eye squint status of the user;
determining a probability of a solar glare exceeding a minimum threshold value, wherein the probability of the solar glare is determined based on the eye squint status of the user and a weighted series of prediction indicators comprising a time index indicative of a time of day and a weight associated with a proximity of the time of day to a present time of day; and
generating, based on the probability of the solar glare and the eye squint status, an AR display of a vehicle path.

2. The method according to claim 1 , further comprising:
determining a vehicle speed;
determining a location of the vehicle and a vehicle trajectory;
selecting, based on the vehicle speed and the vehicle trajectory, a visual comfort mode; and
generating, based on the visual comfort mode, the AR display of the vehicle path.

3. The method according to claim 2 , wherein the visual comfort mode comprises an optical setting for the AR display of the vehicle path comprising a brightness setting.

4. The method according to claim 2 , wherein the visual comfort mode comprises an optical setting for the AR display of the vehicle path comprising a contrast setting.

5. The method according to claim 2 , wherein the visual comfort mode comprises an optical setting for the AR display of the vehicle path comprising a color setting.

6. The method according to claim 1 , wherein the probability of the solar glare is further based on a glare indicator signal from a vehicle vision system and a duration associated with the eye squint status of the user.

7. The method according to claim 1 , wherein the weight is a first weight, and wherein the probability of the solar glare is further based on a second weight associated with a history of solar glare incidences comprising location information indicative of a date and a time associated with the solar glare incidences, a location associated with the solar glare incidences, and a vehicle trajectory associated with the solar glare incidences.

8. A system for providing augmented reality (AR) vehicle guidance for solar glare mitigation in a vehicle, comprising:
a processor; and
a memory for storing executable instructions, the processor programmed to execute the instructions to:
receive, via a camera sensor, an image frame of a user driving the vehicle;
determine, based on the image frame, an eye squint status of the user;
determine a probability of a solar glare exceeding a minimum threshold value, wherein the probability of the solar glare is determined based on the eye squint status of the user and a weighted series of prediction indicators comprising a time index indicative of a time of day and a weight associated with a proximity of the time of day to a present time of day; and
generate, based on the probability of the solar glare and the eye squint status, an AR display of a vehicle path.

9. The system according to claim 8 , wherein the processor is further programmed to execute the instructions to:
determine a vehicle speed;
determine a location of the vehicle and a vehicle trajectory;
select, based on the vehicle speed and the vehicle trajectory, a visual comfort mode; and
generate, based on the visual comfort mode, the AR display of the vehicle path.

10. The system according to claim 9 , wherein the visual comfort mode comprises an optical setting for the AR display of the vehicle path comprising a brightness setting.

11. The system according to claim 9 , wherein the visual comfort mode comprises an optical setting for the AR display of the vehicle path comprising a contrast setting.

12. The system according to claim 9 , wherein the visual comfort mode comprises an optical setting for the AR display of the vehicle path comprising a color setting.

13. The system according to claim 8 , wherein the probability of the solar glare is further based on a glare indicator signal from a vehicle vision system and a duration associated with the eye squint status of the user.

14. The system according to claim 8 , wherein the weight is a first weight, and wherein the probability of the solar glare is further based on a second weight associated with a history of solar glare incidences comprising location information indicative of a date and a time associated with the solar glare incidences, a location associated with the solar glare incidences, and a vehicle trajectory associated with the solar glare incidences.

15. A non-transitory computer-readable storage medium in a vehicle control module, the computer-readable storage medium having instructions stored thereupon which, when executed by a processor, cause the processor to:
receive, via a camera sensor, an image frame of a user driving a vehicle;
determine, based on the image frame, an eye squint status of the user;
determine a probability of a solar glare exceeding a minimum threshold value, wherein the probability of the solar glare is determined based on the eye squint status of the user and a weighted series of prediction indicators comprising a time index indicative of a time of day and a weight associated with a proximity of the time of day to a present time of day; and
generate, based on the probability of the solar glare and the eye squint status, an AR display of a vehicle path.

16. The non-transitory computer-readable storage medium according to claim 15 , having further instructions stored thereupon to:
determine a vehicle speed;
determine a location of the vehicle and a vehicle trajectory;
select, based on the vehicle speed and the vehicle trajectory, a visual comfort mode; and
generate, based on the visual comfort mode, the AR display of the vehicle path.

17. The non-transitory computer-readable storage medium according to claim 16 , wherein the visual comfort mode comprises an optical setting for the AR display of the vehicle path comprising a brightness setting, a color setting, and/or a contrast setting.

18. The non-transitory computer-readable storage medium according to claim 15 , wherein the weight is a first weight, and wherein the probability of the solar glare is further based on a glare indicator signal from a vehicle vision system and a duration associated with the eye squint status of the user, and/or a second weight associated with a history of solar glare incidences comprising location information indicative of a date and a time associated with the solar glare incidences, a location associated with the solar glare incidences, and a vehicle trajectory associated with the solar glare incidences.

Assignees

• Ford Global Tech Llc
• Ford Global Tech L L C

Inventors

• Prakah-Asante Kwaku O
• Wan Jian
• Hegde Prayat

Classifications

• G06V20/597

  Recognising the driver's state or behaviour, e.g. attention or drowsiness · CPC title

• G09G2380/10

  Automotive applications · CPC title

• B60W2540/223

  Posture, e.g. hand, foot, or seat position, turned or inclined · CPC title

• G01C21/365

  Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself · CPC title

• B60W40/105

  Speed · CPC title