Selective rendering of sparse peripheral displays based on user movements

Patent metadata

Field	Value
Publication number	US-10277943-B2
Application number	US-201715470806-A
Country	US
Kind code	B2
Filing date	Mar 27, 2017
Priority date	Mar 27, 2017
Publication date	Apr 30, 2019
Grant date	Apr 30, 2019

Abstract

Peripheral visualizations are based on user movements and/or interactions with elements in a scene. Various user movements are detected while a scene is being rendered. Afterwards, the embodiments determine whether one of the movements corresponds with an increase in a level of focus by the user to the one or more elements and/or an interaction by the user with the one or more elements. Thereafter, peripheral visualizations are rendered on one or more peripheral displays proximate elements that correspond with the user movements/interactions/focus. In some instances, the selective rendering is performed in response to a determination that the user movement does correspond with the increase in the level of focus by the user to the one or more elements and/or the interaction by the user with the one or more elements.

First claim

What is claimed is:

1. A computer system comprising:
a head-mounted display that includes both a primary display and one or more peripheral displays, wherein the one or more peripheral displays include a set of one or more low-resolution light-emitting pixels, and wherein the one or more low-resolution light-emitting pixels have a default configuration of being turned off and are selectively turned on to emit light only upon occurrence of a triggering action;
one or more processors; and
one or more computer-readable hardware storage media having stored thereon computer-executable instructions, the computer-executable instructions being executable by the one or more processors to cause the computer system to selectively render a peripheral visualization with the one or more peripheral displays corresponding to a scene rendered on the primary display based at least in part on a detected movement of a user by causing the computer system to:
render the scene with one or more elements on the primary display;
detect the user movement while the scene is being rendered;
determine whether the user movement is the triggering action by determining whether the user movement corresponds with (1) an increase in a level of focus by the user to the one or more elements or (2) an interaction by the user with the one or more elements; and
selectively trigger at least one of the one or more low-resolution light-emitting pixels to emit light so as to render the peripheral visualization on the one or more peripheral displays responsive to determining the user movement is the triggering action and does correspond with (1) the increase in the level of focus by the user to the one or more elements or (2) the interaction by the user with the one or more elements.

2. The computer system of claim 1 , wherein the user movement is an eye movement corresponding to one or more eyes of the user.

3. The computer system of claim 1 , wherein the user movement is a head movement of the user.

4. The computer system of claim 1 , wherein the interaction by the user includes a gesture of the user.

5. The computer system of claim 1 , wherein the peripheral visualization is a generalized outline of a virtual object.

6. The computer system of claim 1 , wherein selectively rendering the peripheral visualization on the one or more peripheral displays is further based on determining whether the one or more elements include an element that is determined to be salient, the salient determination being based on one or more characteristics of the element, whereby the computer system refrains from rendering the peripheral visualization until after the element is identified as being salient.

7. The computer system of claim 1 , wherein a saliency level of the one or more elements is influenced based on a context associated with the scene such that the saliency level of the one or more elements is scalable according to environmental noise in the scene.

8. One or more hardware storage devices having stored thereon computer-executable instructions, the computer-executable instructions being executable by one or more processors of a computer system to cause the computer system to selectively render a peripheral visualization using one or more peripheral displays corresponding to a scene rendered on a primary display based at least in part on a detected movement of a user by causing the computer system to:
render the scene with one or more elements on the primary display;
detect the user movement while the scene is being rendered;
determine whether the user movement corresponds with (1) an increase in a level of focus by the user to the one or more elements or (2) an interaction by the user with the one or more elements; and
determining one or more characteristics of the one or more elements, the one or more characteristics indicating that the one or more elements are located at an edge region of the scene;
identifying an area of a focus of the user, the area of focus not being near the edge region of the scene;
dynamically altering a saliency level of the one or more elements to thereby reduce a relevancy of the one or more elements in the scene, whereby, by reducing the relevancy of the one or more elements, the user's focus is not diverted from the area of focus; and
selectively render the peripheral visualization on the one or more peripheral displays responsive to determining the user movement corresponds with (1) the increase in the level of focus by the user to the one or more elements or (2) the interaction by the user with the one or more elements.

9. The one or more hardware storage devices of claim 8 , wherein the one or more peripheral displays are disposed around an outer perimeter of the primary display, and wherein the one or more peripheral displays operate in a default off configuration for all pixels until the peripheral visualization is selectively rendered.

10. The one or more hardware storage devices of claim 8 , wherein execution of the computer-executable instructions further causes the computer system to determine a saliency level of the one or more elements.

11. The one or more hardware storage devices of claim 10 , wherein the saliency level is based on a determined historical behavior of the user, the historical behavior being learned by the computer system.

12. The one or more hardware storage devices of claim 11 , wherein the computer system includes environmental understanding cameras, and wherein the computer system at least partially uses the environmental understanding cameras to learn the historical behavior of the user.

13. The one or more hardware storage devices of claim 10 , wherein the saliency level is influenced by characteristics of other elements that are identified as being proximate to the one or more elements in the scene.

14. The one or more hardware storage devices of claim 10 , wherein the saliency level is manually configurable such that the user of the computer system determines the saliency level.

15. The one or more hardware storage devices of claim 10 , wherein the computer system calibrates the saliency level based on one or more of 1) a determined reaction time of the user, 2) a determined historical behavior of the user, or 3) a configuration setting associated with the user.

16. A method for selectively rendering a peripheral visualization using one or more peripheral displays corresponding to a scene rendered on a primary display based at least in part on a detected movement of a user, the method being implemented by one or more processors of a computer system, the method comprising:
rendering the scene with one or more elements on the primary display, wherein the scene further includes one or more navigation buttons;
detecting the user movement while the scene is being rendered, wherein the one or more navigation buttons are selectable by the movement of the user;
determining whether the user movement corresponds with (1) an increase in a level of focus by the user to the one or more elements or (2) an interaction by the user with the one or more elements;
identifying that the one or more elements are at least partially being occluded by the one or more navigation buttons; and
selectively rendering the peripheral visualization on the one or more peripheral displays responsive to determining the user movement corresponds with (1) the increase in the level of focus by the user to the one or more elements or (2) the interaction by the user with the one or more elements, wherein the peripheral visualization are selectively rendered even though the one or more elements are at least partially

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Crisler Matthew Calbraith
• Held Robert Thomas
• Glik Eliezer

Classifications

• G02B27/017

  Head mounted · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title