Immersive virtual display

Patent metadata

Field	Value
Publication number	US-11790616-B2
Application number	US-202217709273-A
Country	US
Kind code	B2
Filing date	Mar 30, 2022
Priority date	Sep 23, 2016
Publication date	Oct 17, 2023
Grant date	Oct 17, 2023

Abstract

A VR system for vehicles that may implement methods that address problems with vehicles in motion that may result in motion sickness for passengers. The VR system may provide virtual views that match visual cues with the physical motions that a passenger experiences. The VR system may provide immersive VR experiences by replacing the view of the real world with virtual environments. Active vehicle systems and/or vehicle control systems may be integrated with the VR system to provide physical effects with the virtual experiences. The virtual environments may be altered to accommodate a passenger upon determining that the passenger is prone to or is exhibiting signs of motion sickness.

First claim

What is claimed is:

1. A system, comprising:
one or more processors; and
a memory storing instructions that, when executed on or across the one or more processors, cause the one or more processors to:
obtain inputs from one or more sources wherein the one or more sources include sensors of a vehicle, wherein the inputs include information about an environment outside of the vehicle, and wherein the environment outside of the vehicle comprises objects;
generate virtual content based at least in part on the inputs, wherein the virtual content comprises:
virtual representations having motions and accelerations that are synchronized with motions and accelerations of the vehicle as indicated in the inputs; and
other virtual representations of the objects in the environment outside of the vehicle, wherein motions and accelerations of the other virtual representations are based on motions and accelerations indicated in the inputs and differ from the motions or accelerations of the virtual representations; and

send the virtual content to a display device for a passenger in the vehicle.

2. The system of claim 1 , further comprising:
a virtual reality (VR) controller configured to render frames according to the virtual content.

3. The system of claim 1 , wherein the inputs include one or more of world map data, 3D models of a local environment, information about objects or features in the local environment, video streams, or audio.

4. The system of claim 1 , wherein the sensors of the vehicle comprise one or more cameras configured to capture a view of the environment.

5. The system of claim 1 , wherein the memory further comprises instructions that, when executed on or across the one or more processors, cause the one or more processors to:
send, to one or more vehicle systems, one or more signals that cause the one or more vehicle systems to provide effects that are synchronized with the virtual content.

6. The system of claim 1 , wherein the virtual content is generated based at least in part on a location of the vehicle.

7. The system of claim 1 , wherein the virtual content comprises audio and video content.

8. A method, comprising:
performing, with one or more computing devices:
obtaining inputs from one or more sources wherein the one or more sources include sensors of a vehicle, wherein the inputs include information about an environment outside of the vehicle, and wherein the environment outside of the vehicle comprises objects;
generating virtual content based at least in part on the inputs, wherein the virtual content comprises:
virtual representations having motions and accelerations that are synchronized with motions and accelerations of the vehicle indicated in the inputs; and
other virtual representations of the objects in the environment outside of the vehicle, wherein motions and accelerations of the other virtual representations are based on motions and accelerations indicated in the inputs and differ from the motions or accelerations of the virtual representations; and

sending the virtual content to a display device for a passenger in the vehicle.

9. The method of claim 8 , further comprising:
anchoring the virtual content with an object in the environment.

10. The method of claim 8 , wherein the sensors of the vehicle comprise a camera configured to capture a view of the environment.

11. The method of claim 8 , further comprising:
sending, to one or more vehicle systems, one or more signals that cause the one or more vehicle systems to provide effects that are synchronized with the virtual content.

12. The method of claim 8 , wherein the virtual content is generated based at least in part on a location of the vehicle.

13. The method of claim 12 , further comprising:
modifying the virtual content based at least in part on a change of the location of the vehicle.

14. The method of claim 8 , wherein the inputs include one or more of world map data, 3D models of a local environment, information about objects or features in the local environment, video streams, or audio.

15. The method of claim 8 , wherein the virtual content comprises audio and video content.

16. The method of claim 8 , further comprising:
altering the virtual content to accommodate the passenger based on determining that the passenger is prone to or is exhibiting signs of motion sickness.

17. One or more non-transitory, computer-readable storage media storing instructions that, when executed on or across one or more processors, cause the one or more processors to:
obtain inputs from one or more sources wherein the one or more sources include sensors of a vehicle, wherein the inputs include information about an environment outside of the vehicle, and wherein the environment outside of the vehicle comprises objects;
generate virtual content based at least in part on the inputs, wherein the virtual content comprises:
virtual representations having motions and accelerations that are synchronized with motions and accelerations of the vehicle indicated in the inputs; and
other virtual representations of the objects in the environment outside of the vehicle, wherein motions and accelerations of the other virtual representations are based on motions and accelerations indicated in the inputs and differ from the motions or accelerations of the virtual representations; and

send the virtual content to a display device for a passenger in the vehicle.

18. The one or more non-transitory, computer-readable storage media of claim 17 , further comprising instructions that, when executed on or across the one or more processors, cause the one or more processors to:
send, to one or more vehicle systems, one or more signals that cause the one or more vehicle systems to provide effects that are synchronized with the virtual content.

19. The one or more non-transitory, computer-readable storage media of claim 17 , further comprising instructions that, when executed on or across the one or more processors, cause the one or more processors to:
generate the virtual content based at least in part on location data for the vehicle.

20. The one or more non-transitory, computer-readable storage media of claim 17 , further comprising instructions that, when executed on or across the one or more processors, cause the one or more processors to:
alter the virtual content to accommodate the passenger based on determining that the passenger is prone to or is exhibiting signs of motion sickness.

Assignees

• Apple Inc

Inventors

• Rober Mark B
• Cohen Sawyer I
• Kurz Daniel
• Holl Tobias
• Lyon Benjamin B
• Meier Peter Georg
• Riepling Jeffrey M
• Gerhard Holly

Classifications

• B60K35/00

  Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles · CPC title

• B60K35/26

  using acoustic output · CPC title

• B60K35/80

  Arrangements for controlling instruments · CPC title

• B60K35/211

  producing three-dimensional [3D] effects, e.g. stereoscopic images · CPC title

• B60K35/25

  using haptic output · CPC title