Augmented reality and virtual reality feedback enhancement system, apparatus and method

Patent metadata

Field	Value
Publication number	US-10964091-B2
Application number	US-201916271368-A
Country	US
Kind code	B2
Filing date	Feb 8, 2019
Priority date	Apr 17, 2017
Publication date	Mar 30, 2021
Grant date	Mar 30, 2021

Abstract

Systems, apparatuses and methods may provide away to render augmented reality and virtual reality (VR/AR) environment information. More particularly, systems, apparatuses and methods may provide a way to selectively suppress and enhance VR/AR renderings of n-dimensional environments. The systems, apparatuses and methods may deepen a user's VR/AR experience by focusing on particular feedback information, while suppressing other feedback information from the environment.

First claim

What is claimed is:

1. An apparatus comprising:
a substrate; and
logic coupled to the substrate, wherein the logic is implemented in one or more of configurable hardware logic or fixed-functionality hardware logic, and the logic is to:
monitor one or more actions of an observer observing n-dimensional environment information, the environment information including feedback information;
detect feedback information, wherein the feedback information includes a priority relationship to the observer, wherein the priority relationship identifies one or more of observer directed feedback information directed to the observer or undirected feedback information;
normalize the environment information and the feedback information; and
render the normalized feedback information based on the one or more actions of the observer and the priority relationship of the feedback information to the observer,
wherein the apparatus is to render at least one of a plurality of sound sources, when detected in a non-focus area, as a first silhouette in the field of view of the observer detected viewing an initial focus area,
wherein the apparatus is to transition the first silhouette to a primary focus area view based on a focus time threshold, and
wherein when the first silhouette transitions to the primary focus area view, the apparatus transitions the initial focus area to a second silhouette.

2. The apparatus of claim 1 , wherein the apparatus attenuates one or more feedback devices based on one or more operating parameters, the feedback devices being coupled to the apparatus.

3. The apparatus of claim 2 ,
wherein one or more of a gaze direction, a focus area, non-focus area or the location of the observer are monitored,
wherein the environment information includes one or more of virtual reality (VR) information or augmented reality (AR) information to render one or more VR/AR scenes,
wherein the one or more of VR information or AR information includes one or more objects, surfaces, sounds, or events,
wherein the operating parameters include one or more of power availability, power consumption parameters, user preferences, application parameters, or performance parameters, and
wherein the environment information includes captured information captured by one or more capture devices located in one or more n-dimensional environments.

4. The apparatus of claim 3 ,
wherein the apparatus is to superimpose over at least one of the VR/AR scenes rendered in the gaze direction, a silhouette of an object, surface, or an event in the direction of the sound or sound source, the event or events source,
wherein quality of the feedback information rendered in one or more of the focus area or the non-focus area is modulated, and
wherein the non-focus area feedback information outside the focus area is attenuated to draw attention or focus of the observer to the non-focus area feedback information.

5. The apparatus of claim 3 ,
wherein the apparatus is to adjust the rendering of the one or more VR/AR scenes based on one or more of the operating parameters,
wherein the apparatus is to adjust a head related transformation function (HRTF) based on one or more of the operating parameters, wherein the feedback devices include one or more beamforming directional microphones, and
wherein the one or more actions include one or more of an eye gaze, gesture, or auditory signal.

6. The apparatus of claim 2 ,
wherein vector paths for the feedback information are calculated using one or more of ambisonics sound field digital signal processing or ambiophonic soundfield digital signal processing, and
wherein one or more of the actions of the observer are predicted.

7. At least one non-transitory computer readable storage medium comprising a set of instructions which, when executed, cause a computing device to:
monitor one or more actions of an observer observing n-dimensional environment information, the environment information including feedback information;
detect feedback information, wherein the feedback information includes a priority relationship to the observer, wherein the priority relationship identifies one or more of observer directed feedback information directed to the observer or undirected feedback information;
normalize the environment information and the feedback information;
render the normalized feedback information based on the one or more actions of the observer and the priority relationship of the feedback information to the observer;
render at least one of a plurality of sound sources, when detected in a non-focus area, as a first silhouette in the field of view of the observer detected viewing an initial focus area; and
transition the first silhouette to a primary focus area view based on a focus time threshold;
wherein when the first silhouette transitions to the primary focus area view, the initial focus area transitions to a second silhouette.

8. The at least one non-transitory computer readable storage medium of claim 7 , wherein the instructions, when executed, further cause the computing device to attenuate one or more feedback devices based on one or more operating parameters, the feedback devices being coupled to the computing device.

9. The at least one non-transitory computer readable storage medium of claim 8 , wherein the instructions, when executed, further cause the computing device to monitor one or more of a gaze direction, a focus area, non-focus area or the location of the observer;
wherein the environment information includes one or more of virtual reality (VR) information or augmented reality (AR) information to render one or more VR/AR scenes,
wherein the one or more of VR information or AR information includes one or more objects, surfaces, sounds, or events,
wherein the operating parameters include one or more of power availability, power consumption parameters, user preferences, application parameters, or performance parameters, and
wherein the environment information includes captured information captured by one or more capture devices located in one or more n-dimensional environments.

10. The at least one non-transitory computer readable storage medium of claim 9 , wherein the instructions, when executed, further cause the computing device to superimpose over at least one of the VR/AR scenes rendered in the gaze direction, a silhouette of an object, surface, or an event in the direction of the sound or sound source, the event or events source;
wherein quality of the feedback information rendered in one or more of the focus area or the non-focus area is modulated, and
wherein the non-focus area feedback information outside the focus area is attenuated to draw attention or focus of the observer to the non-focus area feedback information.

11. The at least one non-transitory computer readable storage medium of claim 9 , wherein the instructions, when executed, further cause the computing device to:
adjust the rendering of the one or more VR/AR scenes based on one or more of the operating parameters; and
adjust a head related transformation function (HRTF) based on one or more of the operating parameters;
wherein the feedback devices include one or more beamforming directional microphones; and
wherein the one or more actions include one or more of an eye gaze, gesture, or auditory signal.

12. The at least one non-transitory computer readable storage medium of claim 8 , wherein the instructions, when executed, further cause the computing device to:
calculate vector paths for the feedback information using one or more of ambisonics sound field digital signal processing or ambiophonic soundfield digital signal processing;
wherein one or

Assignees

• Intel Corp

Inventors

• Sakthivel Chandrasekaran
• Apodaca Michael
• Xiao Kai
• Koker Altug
• Boles Jeffery S
• Lake Adam T
• Kaburlasos Nikos
• Ray Joydeep
• Feit John H
• Schluessler Travis T
• Kwiatkowski Jacek
• Holland James M
• Surti Prasoonkumar
• Kennedy Jonathan
• Feng Louis
• Das Barnan
• Biswal Narayan
• Baran Stanley J
• Cilingir Gokcen
• Shah Nilesh V
• Sharma Archie
• Varerkar Mayuresh M

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06T15/06 Primary

  Ray-tracing · CPC title

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title

• G06T17/10

  Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes · CPC title