Context-aware extended reality systems

Patent metadata

Field	Value
Publication number	US-11922594-B2
Application number	US-202217819049-A
Country	US
Kind code	B2
Filing date	Aug 11, 2022
Priority date	Jan 27, 2021
Publication date	Mar 5, 2024
Grant date	Mar 5, 2024

Abstract

Techniques and systems are provided for dynamically adjusting virtual content provided by an extended reality system. In some examples, a system determines a level of distraction of a user of the extended reality system due to virtual content provided by the extended reality system. The system determines whether the level of distraction of the user due to the virtual content exceeds or is less than a threshold level of distraction, where the threshold level of distraction is determined based at least in part on one or more environmental factors associated with a real world environment in which the user is located. The system also adjusts one or more characteristics of the virtual content based on the determination of whether the level of distraction of the user due to the virtual content exceeds or is less than the threshold level of distraction.

First claim

What is claimed is:

1. An apparatus for adjusting virtual content, the apparatus comprising:
at least one memory; and
at least one processor coupled to the at least one memory, the at least one processor configured to:
determine, for a user of an extended reality system located in a real-world environment, a level of distraction of the user from the real-world environment due to virtual content presented on a display of the extended reality system, wherein the real-world environment is viewable through at least a portion of the display of the extended reality system as the virtual content is displayed;
determine the level of distraction of the user from the real-world environment due to the virtual content exceeds a threshold level of distraction that is determined based on the real-world environment; and
adjust one or more characteristics of the virtual content to emphasize at least one physical object within the real-world environment based on the determination that the level of distraction of the user from the real-world environment due to the virtual content exceeds the threshold level of distraction.

2. The apparatus of claim 1 , wherein the at least one processor is configured to:
determine the threshold level of distraction based at least in part on one or more environmental factors associated with the real-world environment in which the user is located.

3. The apparatus of claim 2 , wherein the one or more environmental factors include at least one of the at least one physical object within the real-world environment or an interaction between the user of the extended reality system and a person.

4. The apparatus of claim 2 , wherein the at least one processor is configured to:
determine the threshold level of distraction further based on determining a level of importance of the one or more environmental factors, a low threshold level of distraction being associated with a high level of importance of the one or more environmental factors.

5. The apparatus of claim 4 , wherein an environmental factor with a high safety risk to the user is associated with a high level of importance.

6. The apparatus of claim 2 , wherein the at least one processor is further configured to:
determine an updated level of distraction of the user of the extended reality system due to the virtual content with the adjusted one or more characteristics; and
determine an updated threshold level of distraction based at least in part on one or more updated environmental factors associated with the real-world environment, the one or more updated environmental factors being at least partially different than the one or more environmental factors.

7. The apparatus of claim 6 , wherein the at least one processor is further configured to:
determine that the updated level of distraction of the user due to the virtual content with the adjusted one or more characteristics exceeds the updated threshold level of distraction; and
further adjust the one or more characteristics of the virtual content based on the determination that the updated level of distraction exceeds the updated threshold level of distraction, wherein further adjusting the one or more characteristics of the virtual content decreases a prominence of the virtual content provided by the extended reality system.

8. The apparatus of claim 6 , wherein the at least one processor is further configured to:
determine that the updated level of distraction of the user due to the virtual content with the adjusted one or more characteristics is less than the updated threshold level of distraction; and
further adjust the one or more characteristics of the virtual content based on determining that the updated level of distraction is less than the updated threshold level of distraction, wherein further adjusting the one or more characteristics of the virtual content increases a prominence of the virtual content provided by the extended reality system.

9. The apparatus of claim 1 , wherein the at least one processor is configured to:
determine the level of distraction of the user of the extended reality system due to the virtual content provided by the extended reality system at least in part by determining a level of engagement between the user and the virtual content.

10. The apparatus of claim 9 , wherein the at least one processor is configured to determine the level of engagement between the user and the virtual content at least in part by monitoring one or more physical behaviors of the user.

11. The apparatus of claim 10 , wherein the one or more physical behaviors of the user indicate at least one of a virtual object or a physical object on which the user is focusing.

12. The apparatus of claim 1 , wherein the at least one processor is configured to:
determine the level of distraction of the user of the extended reality system due to the virtual content provided by the extended reality system at least in part by determining a prominence of the virtual content within the display of the extended reality system, a high prominence being associated with a high level of distraction.

13. The apparatus of claim 12 , wherein the at least one processor is configured to:
determine the prominence of the virtual content within the display of the extended reality system at least in part by determining at least one of a volume of audio of the virtual content, a size of the virtual content, a type of the virtual content, a brightness of the virtual content, a transparency of the virtual content, a level of movement of the virtual content, one or more color characteristics of the virtual content, a vibration intensity of the virtual content, a vibration frequency of the virtual content, a vibration duration of the virtual content, a field of view of occupied by the virtual content, a visual depth of the virtual content, or a position of the virtual content.

14. The apparatus of claim 1 , wherein, to adjust the one or more characteristics of the virtual content to emphasize the at least one physical object in the real-world environment, the at least one processor is configured to output at least one item of virtual content highlighting the at least one physical object.

15. The apparatus of claim 1 , wherein the at least one processor is configured to:
based on the determination that the level of distraction of the user due to the virtual content exceeds the threshold level of distraction, decrease a prominence of the virtual content within the display of the extended reality system.

16. The apparatus of claim 1 , wherein the at least one processor is further configured to:
determine an updated level of distraction of the user of the extended reality system due to the virtual content with the adjusted one or more characteristics;
determine that the updated level of distraction of the user is less than the threshold level of distraction or an updated level of distraction; and
based on the determination that the updated level of distraction of the user is less than the threshold level of distraction or the updated level of distraction, adjust the one or more characteristics of the virtual content by increasing a prominence of the virtual content within the display of the extended reality system.

17. The apparatus of claim 1 , wherein the apparatus is the extended reality system.

18. The apparatus of claim 1 , wherein the apparatus is in communication with the extended reality system.

19. A method of adjusting virtual content, the method comprising:
determining, for a user of an extended reality system located

Assignees

• Qualcomm Inc

Inventors

• Tartz Robert
• Kies Jonathan
• Guest Daniel James

Classifications

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/14

  Digital output to display device {; Cooperation and interconnection of the display device with other functional units} · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06V20/00

  Scenes; Scene-specific elements (control of digital cameras H04N23/60) · CPC title