Systems and methods for in-vehicle driver assistance via augmented reality

Patent metadata

Field	Value
Publication number	US-12311966-B2
Application number	US-202217961967-A
Country	US
Kind code	B2
Filing date	Oct 7, 2022
Priority date	Aug 12, 2022
Publication date	May 27, 2025
Grant date	May 27, 2025

Abstract

The following generally relates to using Augmented Reality (AR) to enhance the in-vehicle experience. In some examples, AR techniques are applied to provide AR indications of vehicle safety indicia to alert vehicle occupants to information that may not otherwise be perceptible. In other example, AR techniques are applied to provide emergency vehicle warnings to improve the likelihood of safe responses thereto. In yet other examples, AR techniques are applied to generated personalized outdoor displays, for example, to ameliorate conditions that may impair safe operation of a vehicle.

First claim

What is claimed:

1. A computer-implemented method for improving vehicle safety via Augmented Reality (AR), the method comprising:
obtaining, via one or more processors of an electronic device on-board a vehicle, environmental data indicative of an environment proximate to the vehicle;
analyzing, via the one or more processors, the environmental data to determine a location of a vehicle safety indicia relative to the vehicle;
determining, via the one or more processors, a field of view of an occupant of the vehicle associated with an AR viewer by:
obtaining, via the one or more processors, image data generated by an image sensor configured to have a field of view oriented within the vehicle, and
based upon the image data, determining, via the one or more processors, a location of the occupant within the vehicle; and

based upon a comparison of the location of the vehicle safety indicia and the field of view of the occupant, presenting, via the one or more processors, an indication of the vehicle safety indicia via the AR viewer.

2. The computer-implemented method of claim 1 , wherein the AR viewer is at least one of a smart windshield, a smart window, or a smart mirror of the vehicle.

3. The computer-implemented method of claim 1 , wherein the AR viewer is a wearable AR viewer.

4. The computer-implemented method of claim 3 , wherein determining the field of view comprises:
obtaining, via the one or more processors, orientation data from the AR viewer.

5. The computer-implemented method of claim 1 , wherein the electronic device on-board the vehicle is an on-board computing system operatively coupled to a sensor system of the vehicle.

6. The computer-implemented method of claim 5 , wherein the environmental data includes at least one of image data or light detection and ranging (LIDAR) data generated via the sensor system of the vehicle.

7. The computer-implemented method of claim 6 , further comprising:
based upon the environmental data, determining, via the one or more processors, a distance between the vehicle and the vehicle safety indicia; and
based upon the distance, scaling, by the one or more processors, a size of the indication of the vehicle safety indicia.

8. The computer-implemented method of claim 1 , wherein obtaining the environmental data comprises:
receiving, from an electronic device on-board a second vehicle, the environmental data.

9. The computer-implemented method of claim 8 , wherein the environmental data includes at least one of (i) image data or light detection and ranging (LIDAR) data generated via a sensor system of the second vehicle, (ii) position data of the second vehicle, and (iii) telematics data generated by the sensor system of the second vehicle.

10. The computer-implemented method of claim 9 , wherein presenting the indication of vehicle safety indicia comprises:
presenting, via the one or more processors, the image data generated by the second vehicle.

11. The computer-implemented method of claim 9 , wherein presenting the indication of vehicle safety indicia comprises:
based upon the telematics data, determining, via the one or more processors, a trajectory of the second vehicle; and
presenting, via the one or more processors, an indication of the trajectory of the second vehicle.

12. The computer-implemented method of claim 1 , further comprising:
analyzing, via the one or more processors, the environmental data to detect at least one of (i) the environment being a low-visibility environment, or (ii) a presence of an obscured hazard.

13. The computer-implemented method of claim 12 , wherein the vehicle safety indicia is at least one of a presence of ice, a vehicle, a road marking, a road sign, a pot hole, and a pedestrian.

14. The computer-implemented method of claim 12 , further comprising:
determining, via the one or more processors, a maneuver that indicates safe operation of the vehicle in view of the vehicle safety indicia; and
presenting, via the one or more processors, an indication of the maneuver via the AR viewer.

15. The computer-implemented method of claim 1 , further comprising:
obtaining, via the one or more processors, navigation instructions associated with the vehicle; and
presenting, via the one or more processors, an indication of a navigation instruction via the AR viewer.

16. The computer-implemented method of claim 1 , wherein obtaining the environmental data comprises:
detecting, via the one or more processors, a communication signal emitted by a beacon device, wherein the communication signal includes an indication of the vehicle safety indicia.

17. The computer-implemented method of claim 1 , wherein obtaining the environmental data comprises:
obtaining, via the one or more processors, mapping data from a mapping service provider.

18. A system for improving vehicle safety via Augmented Reality (AR), the system comprising:
one or more processors of an electronic device on-board a vehicle; and
one or more non-transitory memories storing processor-executable instructions that, when executed by the one or more processors, cause the system to:
obtain environmental data indicative of an environment proximate to the vehicle;
analyze the environmental data to determine a location of a vehicle safety indicia relative to the vehicle;
determine a field of view of an occupant of the vehicle associated with an AR viewer by:
obtaining image data generated by an image sensor configured to have a field of view oriented within the vehicle, and
based upon the image data, determining a location of the occupant within the vehicle; and

based upon a comparison of the location of the vehicle safety indicia and the field of view of the occupant, present an indication of the vehicle safety indicia via the AR viewer.

19. The system of claim 18 , wherein the environmental data includes at least one of image data or light detection and ranging (LIDAR) data generated via a sensor system of the vehicle.

20. The system of claim 19 , wherein the instructions, when executed, cause the system to:
based upon the environmental data, determine a distance between the vehicle and the vehicle safety indicia; and
based upon the distance, scale a size of the indication of the vehicle safety indicia.

21. The system of claim 18 , wherein to obtain the environmental data, the instructions, when executed, cause the system to:
receive, from an electronic device on-board a second vehicle, the environmental data.

22. The system of claim 21 , wherein the environmental data includes at least one of (i) image data or light detection and ranging (LIDAR) data generated via a sensor system of the second vehicle, (ii) position data of the second vehicle, and (iii) telematics data generated by the sensor system of the second vehicle.

23. The system of claim 22 , wherein to present the indication of vehicle safety indicia, the instructions, when executed, cause the system to:
present the image data generated by the second vehicle.

24. The system of claim 18 , wherein the instructions, when executed, cause the system to:
analyze the environmental data to detect at least one of (i) the environment being a low-visibility environment, or (ii) a presence of an obscured hazard.

25. The system of claim 24 , wherein the vehicle safety indicia is at least one of a presence of ice, a vehicle, a road marking, a road sign, a pot hole, and a pedestrian.

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Fields Brian M
• Tofte Nathan L
• Williams Aaron C
• Harr Joseph P
• King Vicki

Classifications

• B60K35/00

  Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles · CPC title

• B60K35/235

  with means for detecting the driver's gaze direction or eye points · CPC title

• B60K35/233

  controlling the size or position in display areas of virtual images depending on the condition of the vehicle or the driver · CPC title

• B60K35/81

  for controlling displays · CPC title

• B60K35/29

  Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions · CPC title