Transitioning between imagery and sounds of a virtual environment and a real environment

Patent metadata

Field	Value
Publication number	US-11880911-B2
Application number	US-201917272025-A
Country	US
Kind code	B2
Filing date	Sep 6, 2019
Priority date	Sep 7, 2018
Publication date	Jan 23, 2024
Grant date	Jan 23, 2024

Abstract

The present disclosure relates to techniques for transitioning between imagery and sounds of two different environments, such as a virtual environment and a real environment. A view of a first environment and audio associated with the first environment are provided. In response to detecting a transition event, a view of the first environment combined with a second environment is provided. The combined view includes imagery of the first environment at a first visibility value and imagery of the second environment at a second visibility value. In addition, in response to detecting a transition event, the first environment audio is mixed with audio associated with the second environment.

First claim

What is claimed is:

1. An electronic device, comprising:
one or more processors; and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
providing a view of a first environment of a first type;
providing first environment audio associated with the first environment;
detecting a transition event, wherein detecting the transition event comprises receiving a signal from an input device; and
in response to detecting the transition event:
providing a combined view of the first environment with a second environment of a second type, the combined view including imagery of the first environment at a first visibility value and imagery of the second environment at a second visibility value, wherein the first and second visibility values are based on the received signal from the input device; and
providing a combined mix of the first environment audio with second environment audio associated with the second environment, wherein at least one of the first environment audio or the second environment audio includes a plurality of audio objects, and wherein volume levels of the plurality of audio objects are associated with at least one of the first visibility value or the second visibility value.

2. The electronic device of claim 1 , wherein the first environment of the first type is a virtual environment, and the second environment of the second type is a representation of a real environment.

3. The electronic device of claim 1 , wherein the first environment of the first type is a representation of a real environment, and the second environment of the second type is a virtual environment.

4. The electronic device of claim 1 , wherein providing the combined view comprises compositing the imagery of the first environment with the imagery of the second environment, wherein the compositing uses alpha channels associated with the first environment and the second environment.

5. The electronic device of claim 1 , wherein the first environment audio comprises the plurality of audio objects, and wherein providing the combined mix comprises cross-fading the plurality of audio objects with the second environment audio in accordance with the respective volume levels of the plurality of audio objects.

6. The electronic device of claim 1 , wherein detecting the transition event comprises detecting a triggering sound.

7. The electronic device of claim 6 , wherein the imagery of the second environment includes a representation of the source of the triggering sound.

8. The electronic device of claim 1 , wherein the one or more programs further include instructions for:
after providing the combined view of the first environment with the second environment and the combined mix of the first environment audio with the second environment audio:
ceasing to provide the combined view of the first environment with the second environment and the combined mix of the first environment audio with the second environment audio;
providing a view of the second environment; and
providing second environment audio.

9. The electronic device of claim 1 , wherein the one or more programs further include instructions for:
detecting a second transition event; and
in response to detecting the second transition event:
ceasing to provide the combined view of the first environment with the second environment and the combined mix of the first environment audio with the second environment audio;
providing a view of the second environment; and
providing second environment audio.

10. The electronic device of claim 1 , wherein the one or more programs further include instructions for:
detecting a second transition event; and
in response to detecting the second transition event:
ceasing to provide the combined view of the first environment with the second environment and the combined mix of the first environment audio with the second environment audio;
providing a view of the first environment; and
providing the first environment audio.

11. An electronic device, comprising:
one or more processors; and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
providing a view of a first environment of a first type;
providing first environment audio associated with the first environment;
detecting a transition event, wherein detecting the transition event comprises detecting that a proximity of a user to a physical object in a real environment is less than a threshold distance; and
in response to detecting the transition event:
providing a combined view of the first environment with a second environment of a second type, the combined view including imagery of the first environment at a first visibility value and imagery of the second environment at a second visibility value, wherein the first and second visibility values are based on the proximity of the user to the physical object; and
providing a combined mix of the first environment audio with second environment audio associated with the second environment, wherein at least one of the first environment audio or the second environment audio includes a plurality of audio objects, and wherein volume levels of the plurality of audio objects are associated with at least one of the first visibility value or the second visibility value.

12. The electronic device of claim 11 , wherein the imagery of the second environment includes a representation of the physical object.

13. An electronic device, comprising:
one or more processors; and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
providing a view of a first environment of a first type;
providing first environment audio associated with the first environment;
detecting a transition event; and
in response to detecting the transition event:
providing a combined view of the first environment with a second environment of a second type, the combined view including imagery of the first environment at a first visibility value and imagery of the second environment at a second visibility value, wherein providing the combined view includes aligning a user perspective of the first environment with a user perspective of the second environment; and
providing a combined mix of the first environment audio with second environment audio associated with the second environment, wherein at least one of the first environment audio or the second environment audio includes a plurality of audio objects, and wherein volume levels of the plurality of audio objects are associated with at least one of the first visibility value or the second visibility value.

14. The electronic device of claim 13 , wherein detecting the transition event comprises receiving a signal from an input device, and wherein the first and second visibility values are based on the received signal.

15. The electronic device of claim 13 , wherein detecting the transition event comprises detecting that a proximity of a user to a physical object in a real environment is less than a threshold distance, and wherein the first and second visibility values are based on the proximity of the user to the physical object.

16. The electronic device of claim 13 , wherein providing the combined view comprises compositing the imagery of the first environment with the imagery of the second environment, wherein the compositing uses alpha channels associated with the first environment and the second environ

Assignees

• Apple Inc

Inventors

• Nepveu Bertrand
• Carter Sandy J
• Chapdelaine-Couture Vincent
• Chenier Marc-Andre
• Cote Yan
• Fortin-Deschênes Simon
• Ghannoum Anthony
• Holman Tomlinson
• Lepage Marc-Olivier
• Millette Yves

Classifications

• A63F13/655

  by importing photos, e.g. of the player · CPC title

• A63F13/25

  Output arrangements for video game devices · CPC title

• A63F13/213

  comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) · CPC title

• A63F13/20

  Input arrangements for video game devices · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title