Dynamic content presentation for extended reality systems

Patent metadata

Field	Value
Publication number	US-11934572-B2
Application number	US-202117454188-A
Country	US
Kind code	B2
Filing date	Nov 9, 2021
Priority date	Nov 9, 2021
Publication date	Mar 19, 2024
Grant date	Mar 19, 2024

Abstract

Systems and techniques are described for extended reality (XR) operations. An XR system displays virtual content using a display according to display settings. The display settings can identify, for instance, a position, orientation, and/or size of the virtual content as displayed. The environment can be viewable using the display as the virtual content is displayed by the display, for example using a see-through display or a pass-through display. The imaging system can determine, based on one or more attributes of one or both eyes of the user of the imaging system, an extent of perception of the virtual content that is displayed using the display by the user. The attributes can identify, for instance, eye position, eye movement, pupil dilation, saccades, fixations, blinking, and/or squinting. The XR system can determine, based on the extent of perception of the virtual content by the user, a modification to the display settings.

First claim

What is claimed is:

1. An extended reality (XR) system comprising:
at least one memory; and
at least one processor coupled to the at least one memory, the at least one processor configured to:
cause virtual content to be displayed according to display settings associated with the virtual content, wherein an environment is viewable using the display as the virtual content is displayed by the display, and wherein the virtual content has a level of complexity;
track a gaze of a user relative to the virtual content as the virtual content is displayed to determine a duration of the gaze of the user relative to the virtual content;
determine, based on at least a comparison between the duration of the gaze of the user relative to the virtual content and at least one duration threshold, a level of comprehension by the user of the virtual content that is displayed, wherein the at least one duration threshold is based on the level of complexity of the virtual content; and
determine, based on the level of comprehension by the user of the virtual content, a modification to the display settings is configured to change how prominently the virtual content is configured to be displayed.

2. The XR system of claim 1 , the at least one processor configured to:
determine, based on the gaze of the user relative to the virtual content as tracked, at least one perception-related attribute of the user that is associated with at least one eye of the user.

3. The XR system of claim 1 , wherein the environment is viewable using the display at least in part based on light from the environment passing through at least a portion of the display.

4. The XR system of claim 1 , wherein the environment is viewable at least in part based on the at least one processor being configured to cause a view of the environment to be displayed by the display.

5. The XR system of claim 1 , wherein the at least one processor is configured to use at least one perception-related attribute of the user as at least one input to at least one trained machine learning model to determine the level of comprehension by the user of the virtual content, the at least one perception-related attribute of the user being associated with the gaze of the user relative to the virtual content as tracked.

6. The XR system of claim 5 , the at least one processor configured to:
receive, through a user interface, feedback corresponding to the level of comprehension by the user of the virtual content; and
update the at least one trained machine learning model based on the feedback.

7. The XR system of claim 1 , the at least one processor configured to:
receive sensor data captured by sensors at least one sensor, wherein the sensor data is indicative of at least one eye of the user; and
determine the duration of the gaze of the user relative to the virtual content based on the sensor data.

8. The XR system of claim 7 , further comprising:
the at least one sensor.

9. The XR system of claim 1 , the at least one processor configured to:
determine the duration of the gaze of the user relative to the virtual content based on at least one image of at least one eye of the user captured using at least one image sensor to determine the duration of the gaze.

10. The XR system of claim 1 , the at least one processor configured to:
determine a level of perception by the user of the virtual content to determine the level of comprehension by the user of the virtual content.

11. The XR system of claim 10 , the at least one processor configured to:
receive historical information associated with the user, wherein, to determine the level of comprehension of the virtual content by the user, the at least one processor is configured to determine the level of comprehension of the virtual content by the user based on the historical information about the user.

12. The XR system of claim 1 , the at least one processor configured to:
determine a characteristic of the virtual content, wherein, to determine the level of perception of the virtual content by the user, the at least one processor is configured to determine the level of perception based further on the characteristic of the virtual content.

13. The XR system of claim 1 , the at least one processor configured to:
determine the level of complexity of the virtual content.

14. The XR system of claim 1 , the at least one processor configured to:
determine a level of similarity of the virtual content to previously-displayed virtual content; and
determine the level of comprehension by the user based on the level of similarity of the virtual content to the previously-displayed virtual content to determine the level of comprehension by the user of the virtual content.

15. The XR system of claim 1 , wherein the modification to the display settings corresponding to the virtual content is configured to cause the display to stop displaying at least a portion of the virtual content.

16. The XR system of claim 1 , wherein the modification to the display settings is configured to cause the display to display at least a portion of the virtual content more prominently than the virtual content is configured to be displayed before the modification.

17. The XR system of claim 1 , wherein the modification to the display settings corresponding to the virtual content is configured to cause a modification to at least one characteristic of the virtual content, wherein the at least one characteristic includes at least one of a position, an orientation, a depth, a size, a color, a font size, a font color, a font, a language, or a layout.

18. The XR system of claim 1 , the at least one processor configured to determine that the user has perceived the virtual content to determine the level of comprehension by the user of the virtual content.

19. The XR system of claim 1 , the at least one processor configured to determine that the user has not perceived the virtual content to determine the level of comprehension by the user of the virtual content.

20. The XR system of claim 1 , wherein the modification to the display settings is based on a likelihood that the virtual content is to be reviewed by the user in a certain amount of time.

21. The XR system of claim 1 , the at least one processor configured to determine a confidence level corresponding to the level of perception comprehension by the user of the virtual content to determine the level of comprehension by the user of the virtual content, wherein the modification to the display settings is based on the confidence level.

22. The XR system of claim 2 , wherein the at least one perception-related attribute of the user includes at least one eye position of the at least one eye of the user relative to the virtual content.

23. The XR system of claim 2 , wherein the at least one perception-related attribute of the user includes at least one characteristic of at least one saccade by the at least one eye of the user, wherein the at least one characteristic includes at least one of a frequency, a duration, a timing, a saccade speed, a saccade amplitude, an eye position, or an eye movement.

24. The XR system of claim 2 , wherein the at least one perception-related attribute of the user includes at least one characteristic of at least one fixation by the at least one eye of the user, wherein the at least one characteristic includes at least one of a frequency, a duration, a timing, an eye position, or an eye movement.

Assignees

• Qualcomm Inc

Inventors

• Forutanpour Bijan
• Kies Jonathan

Classifications

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06V40/19

  Sensors therefor · CPC title

• G06V20/20

  in augmented reality scenes · CPC title