Display resource management

Patent metadata

Field	Value
Publication number	US-9851787-B2
Application number	US-201213689379-A
Country	US
Kind code	B2
Filing date	Nov 29, 2012
Priority date	Nov 29, 2012
Publication date	Dec 26, 2017
Grant date	Dec 26, 2017

Abstract

A system and related methods for a resource management in a head-mounted display device are provided. In one example, the head-mounted display device includes a plurality of sensors and a display system for presenting holographic objects. A resource management program is configured to operate a selected sensor in a default power mode to achieve a selected fidelity. The program receives user-related information from one or more of the sensors, and determines whether target information is detected. Where target information is detected, the program adjusts the selected sensor to operate in a reduced power mode that uses less power than the default power mode.

First claim

The invention claimed is:

1. A resource management system, comprising:
a head-mounted display device configured to be worn by a user and operatively connected to a computing device, the head-mounted display device including a plurality of sensors and a display system for presenting holographic objects, and
a resource management program executed by a processor of the computing device, the resource management program configured to:
operate a selected sensor of the plurality of sensors in a default power mode to achieve a selected level of sensor fidelity;
receive user-related information from one or more of the plurality of sensors, the user-related information selected from the group consisting of audio information, user gaze information, user location information, user movement information, user image information, and user physiological information;
determine whether target information is detected in the user-related information; and
when the target information is detected, among the plurality of sensors adjust only the selected sensor to operate in a reduced power mode that uses less power than the default power mode, thereby achieving a reduced level of sensor fidelity;

wherein the plurality of sensors includes an outward facing image sensor, an eye-tracking sensor, and a position sensor, and
wherein adjusting only the selected sensor to operate in the reduced power mode includes:
in response to detecting the target information in the user-related information received from the outward facing image sensor, one or more of reducing a refresh rate and a resolution of the outward facing image sensor,
in response to detecting the target information received from the eye-tracking sensor, reducing a refresh rate of the eye-tracking sensor, and
in response to detecting the target information received from the position sensor, reducing a sampling rate of the position sensor,

wherein the resource management system is configured to override the resource management program and adjust the selected sensor to operate in a fast response mode in response to a transition of the head-mounted display device into a new environment, the fast response mode using more power than the reduced power mode, thereby achieving an increased level of sensor fidelity.

2. The resource management system of claim 1 , wherein the target information is selected from the group consisting of context-identifying audio information, a user gaze that is fixed on one of the holographic objects or the physical object, a user location at a predetermined location, movement of the user below a movement threshold, image information indicating a user body part, and awareness-related user physiological information that indicates a user awareness below an awareness threshold.

3. The resource management system of claim 1 , wherein the plurality of sensors further includes a microphone.

4. The resource management system of claim 1 , wherein the reduced power mode further comprises a powered-off state.

5. The resource management system of claim 1 , wherein the resource management program is further configured to:
receive physical environment information from one or more of the plurality of sensors; and
determine whether the target information is detected in the physical environment information.

6. The resource management system of claim 5 , wherein the physical environment information comprises physical object image data, and the resource management program is further configured to:
determine whether the outward facing image sensor has captured a threshold amount of the physical object image data; and
when the threshold amount of the physical object image data has been captured, adjust the outward facing image sensor to operate in the reduced power mode.

7. The resource management system of claim 1 , wherein the resource management program is further configured to:
receive physical environment information via a network from an external source that is external to the head-mounted display device; and
based on receiving the physical environment information from the external source, adjust the selected sensor to operate in the reduced power mode.

8. The resource management system of claim 1 , wherein the plurality of sensors comprises a microphone, and the resource management program is further configured to:
operate the microphone at a slower polling frequency while target audio is not detected; and
adjust the microphone to operate at a faster polling frequency that is greater than the slower polling frequency when the target audio is detected.

9. The resource management system of claim 1 , wherein the target information comprises a user gaze at one of the holographic objects or a physical object, and the resource management program is further configured to:
detect the user gaze at one of the holographic objects or the physical object using the eye-tracking sensor; and
based on detecting the user gaze, reduce an illumination output of the display system of the head-mounted display device.

10. The resource management system of claim 1 , wherein the target information comprises movement of the user below a movement threshold, and the resource management program is further configured to:
detect that the movement of the user is below the movement threshold using the position sensor; and
based on detecting the movement of the user below the movement threshold, adjust the position sensor to operate in the reduced power mode.

11. A method for managing resources in a head-mounted display device, comprising:
providing the head-mounted display device configured to be worn by a user and operatively connected to a computing device, the head-mounted display device including a plurality of sensors and a display system for presenting holographic objects;
operating a selected sensor of the plurality of sensors in a default power mode to achieve a selected level of sensor fidelity;
receiving user-related information from one or more of the plurality of sensors, the user-related information selected from the group consisting of audio information, user gaze information, user location information, user movement information, user image information, and user physiological information;
determining whether target information is detected in the user-related information;
when the target information is detected, among the plurality of sensors adjusting only the selected sensor to operate in a reduced power mode that uses less power than the default power mode, thereby achieving a reduced level of sensor fidelity; and
adjusting the selected sensor to operate in a fast response mode in response to a transition of the head-mounted display device into a new environment, the fast response mode using more power than the reduced power mode, thereby achieving an increased level of sensor fidelity;
wherein the plurality of sensors includes an outward facing image sensor, an eye-tracking sensor, and a position sensor, and
wherein adjusting only the selected sensor to operate in the reduced power mode includes:
in response to detecting the target information in the user-related information received from the outward facing image sensor, one or more of reducing a refresh rate and a resolution of the outward facing image sensor,
in response to detecting the target information received from the eye-tracking sensor, reducing a refresh rate of the eye-tracking sensor, and
in response to detecting the target information received from the position sensor, reducing a sampling rate of the position sensor.

12. The method for managing resources of claim 11 , wherein the target information is selected from the group consisting of context

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Latta Stephen
• Perry Jedd Anthony
• Fleck Rod G
• Clevenger Jack
• Schaffalitzky Frederik
• Steedly Drew
• Mcculloch Daniel
• Mcintyre Ian
• Balan Alexandru
• Sugden Ben
• Hastings Ryan
• Mount Brian

Classifications

• G06F1/3287

  by switching off individual functional units in the computer system · CPC title

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G06F1/3215

  Monitoring of peripheral devices · CPC title