Augmented reality and virtual reality feedback enhancement system, apparatus and method

Patent metadata

Field	Value
Publication number	US-11514639-B2
Application number	US-202117209412-A
Country	US
Kind code	B2
Filing date	Mar 23, 2021
Priority date	Apr 17, 2017
Publication date	Nov 29, 2022
Grant date	Nov 29, 2022

Abstract

Systems, apparatuses and methods may provide away to render augmented reality and virtual reality (VR/AR) environment information. More particularly, systems, apparatuses and methods may provide a way to selectively suppress and enhance VR/AR renderings of n-dimensional environments. The systems, apparatuses and methods may deepen a user's VR/AR experience by focusing on particular feedback information, while suppressing other feedback information from the environment.

First claim

What is claimed is:

1. A computing system, comprising:
a processor; and
a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the computing system to:
detect sound feedback information in environment information corresponding to a three-dimensional (3D) environment, the environment information including one or more of virtual reality information or augmented reality information (VR/AR) to render one or more VR/AR scenes;
calculate vector paths for the sound feedback information;
receive focus information for an observer observing the 3D environment;
render a sound source, when detected in a non-focus area of the observer, as a first silhouette in a field of view of the observer, the field of view including an initial focus area; and
adjust a head related transformation function (HRTF) based on one or more of an operating parameter or an observer action, the observer action including one or more of an eye gaze rotation, a gesture, or an auditory signal.

2. The computing system of claim 1 , wherein the vector paths for the sound feedback information are to be calculated via a ray tracing engine.

3. The computing system of claim 1 , wherein the vector paths for the sound feedback information are to be calculated by using one or more of ambisonics sound field digital signal processing or ambiophonic sound field digital signal processing.

4. The computing system of claim 1 , wherein the instructions, when executed, further cause the computing system to:
receive information identifying a focus area based on a gaze of the observer; and
transition the first silhouette to a primary focus area view based on a focus time threshold.

5. The computing system of claim 4 , wherein the instructions, when executed, further cause the computing system to transition the initial focus area to a second silhouette when the first silhouette transitions to the primary focus area view.

6. The computing system of claim 1 , wherein a location to render one or more sounds is to be defined by one or more of an adjustment to the HRTF, observer head rotation information, or observer gaze information.

7. A graphics apparatus comprising:
memory; and
logic communicatively coupled to the memory, the logic implemented in one or more of configurable hardware logic or fixed-functionality hardware logic, the logic to:
detect sound feedback information in environment information corresponding to a three-dimensional (3D) environment, the environment information including one or more of virtual reality information or augmented reality information (VR/AR) to render one or more VR/AR scenes;
calculate vector paths for the sound feedback information;
receive focus information for an observer observing the 3D environment;
render a sound source, when detected in a non-focus area of the observer, as a first silhouette in a field of view of the observer; and
adjust a head related transformation function (HRTF) based on one or more of an operating parameter or an observer action, the observer action including one or more of an eye gaze rotation, a gesture, or an auditory signal.

8. The graphics apparatus of claim 7 , wherein the vector paths for the sound feedback information are to be calculated via a ray tracing engine.

9. The graphics apparatus of claim 7 , wherein the vector paths for the sound feedback information are to be calculated by using one or more of ambisonics sound field digital signal processing or ambiophonic sound field digital signal processing.

10. The graphics apparatus of claim 7 , wherein the logic is further to:
receive information identifying a focus area based on a gaze of the observer; and
transition the first silhouette to a primary focus area view based on a focus time threshold.

11. The graphics apparatus of claim 10 , wherein the logic is further to transition the initial focus area to a second silhouette when the first silhouette transitions to the primary focus area view.

12. The graphics apparatus of claim 7 , wherein a location to render one or more sounds is to be defined by one or more of an adjustment to the HRTF, observer head rotation information, or observer gaze information.

13. At least one non-transitory computer readable storage medium comprising a set of instructions which, when executed, cause a computing device to:
detect sound feedback information in environment information corresponding to a three-dimensional (3D) environment, the environment information including one or more of virtual reality information or augmented reality information (VR/AR) to render one or more VR/AR scenes;
calculate vector paths for the sound feedback information;
receive focus information for an observer observing the 3D environment;
render a sound source, when detected in a non-focus area of the observer, as a first silhouette in a field of view of the observer; and
adjust a head related transformation function (HRTF) based on one or more of an operating parameter or an observer action, the observer action including one or more of an eye gaze rotation, a gesture, or an auditory signal.

14. The at least one non-transitory computer readable storage medium of claim 13 , wherein the vector paths for the sound feedback information are to be calculated via a ray tracing engine.

15. The at least one non-transitory computer readable storage medium of claim 13 , wherein the vector paths for the sound feedback information are to be calculated by using one or more of ambisonics sound field digital signal processing or ambiophonic sound field digital signal processing.

16. The at least one non-transitory computer readable storage medium of claim 13 , wherein the instructions, when executed, further cause the computing device to:
receive information identifying a focus area based on a gaze of the observer; and
transition the first silhouette to a primary focus area view based on a focus time threshold.

17. The at least one non-transitory computer readable storage medium of claim 16 , wherein the instructions, when executed, further cause the computing device to transition the initial focus area to a second silhouette when the first silhouette transitions to the primary focus area view.

18. The at least one non-transitory computer readable storage medium of claim 13 , wherein a location to render one or more sounds is to be defined by one or more of an adjustment to the HRTF, observer head rotation information, or observer gaze information.

19. A method comprising:
detect sound feedback information in environment information corresponding to a three-dimensional (3D) environment, the environment information including one or more of virtual reality information or augmented reality information (VR/AR) to render one or more VR/AR scenes;
calculate vector paths for the sound feedback information;
receive focus information for an observer observing the 3D environment;
render a sound source, when detected in a non-focus area of the observer, as a first silhouette in a field of view of the observer; and
adjust a head related transformation function (HRTF) based on one or more of an operating parameter or an observer action, the observer action including one or more of an eye gaze rotation, a gesture, or an auditory signal.

20. The method of claim 19 , wherein the vector paths for the sound feedback information are calculated via a ray tracing engine.

21. The method of claim 19 , wherein the vector paths for the sound feedback information

Assignees

• Intel Corp

Inventors

• Sakthivel Chandrasekaran
• Apodaca Michael
• Xiao Kai
• Koker Altug
• Boles Jeffery S
• Lake Adam T
• Kaburlasos Nikos
• Ray Joydeep
• Feit John H
• Schluessler Travis T
• Kwiatkowski Jacek
• Holland James M
• Surti Prasoonkumar
• Kennedy Jonathan
• Feng Louis
• Das Barnan
• Biswal Narayan
• Baran Stanley J
• Cilingir Gokcen
• Shah Nilesh V
• Sharma Archie
• Varerkar Mayuresh M

Classifications

• H04S2420/01

  Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] · CPC title

• H04S2400/15

  Aspects of sound capture and related signal processing for recording or reproduction · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• H04S2400/01

  Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title