Gaze target application launcher

Patent metadata

Field	Value
Publication number	US-10248192-B2
Application number	US-201414559841-A
Country	US
Kind code	B2
Filing date	Dec 3, 2014
Priority date	Dec 3, 2014
Publication date	Apr 2, 2019
Grant date	Apr 2, 2019

Abstract

A gaze target is recognized via an eye tracking camera. An application launcher is displayed, via a display, at the gaze target based on a user trigger. The application launcher presents a plurality of applications selectable for launching. A user selection of one of the plurality of applications is recognized. The application launcher is replaced with the selected one of the plurality of applications at the gaze target via the display.

First claim

The invention claimed is:

1. A method, comprising:
displaying, via a see-through display of a head-mounted display device, one or more augmentation elements;
recognizing, via an eye-tracking camera of the head-mounted display device, a gaze target on a surface in a real-world environment viewed through the see-through display;
responsive to a user trigger, displaying, via the see-through display, an application launcher overlaid on the surface of the real-world environment at the gaze target, the application launcher having a world-locked position relative to the real-world environment such that the application launcher does not appear to move relative to the real-world environment when the head-mounted display device moves, the application launcher presenting a plurality of applications selectable for launching within a graphical boundary of the application launcher that occupies a region of the see-through display to allow for the one or more augmentation elements and one or more real-world objects in the real-world environment to be viewed outside of the graphical boundary of the application launcher through the see-through display;
recognizing a user selection of one of the plurality of applications; and
replacing, via the see-through display, the application launcher with the selected one of the plurality of applications by displaying a graphical user interface of the selected one of the plurality of applications at the world-locked position overlaid on the surface of the real-world environment where the application launcher was previously displayed such that the graphical user interface of the selected one of the plurality of applications is presented within the graphical boundary of the application launcher.

2. The method of claim 1 , wherein the user trigger comprises a gesture recognized via one or more of the eye-tracking camera and another imaging device of the head-mounted display device.

3. The method of claim 1 , wherein the user trigger comprises a voice command recognized via a microphone of the head-mounted display device.

4. The method of claim 1 , wherein the user trigger comprises a sustained gaze, the method further comprising recognizing the user trigger responsive to recognizing, via the eye-tracking camera, that a user gaze is directed to the gaze target for a duration that is greater than a trigger threshold.

5. The method of claim 1 , further comprising, prior to displaying the application launcher, displaying, via the see-through display, a positional preview of the graphical boundary of the application launcher overlaid on the surface of the real-world environment at the gaze target, the positional preview providing an indication of position, size, and/or shape of the graphical boundary within which the application launcher will be displayed.

6. The method of claim 5 , further comprising replacing, via the see-through display, the positional preview of the graphical boundary of the application launcher with the application launcher presenting the plurality of applications selectable for launching within the graphical boundary of the application launcher responsive to a confirmation input.

7. The method of claim 1 , wherein the application launcher is displayed, via the see-through display, with a center point of the application launcher positioned at the gaze target on the surface of the real-world environment, and wherein the graphical user interface of the selected one of the plurality of applications is displayed, via the see-through display, with a center point of the graphical user interface positioned at a same location where the center point of the application launcher was previously displayed.

8. A head-mounted display device comprising:
an eye-tracking camera;
a see-through display;
a logic machine; and
a storage machine holding instructions executable by the logic machine to:
display, via the see-through display, one or more augmentation elements;
recognize, via the eye-tracking camera, a gaze target on a surface in a real-world environment viewed through the see-through display;
responsive to a user trigger, display, via the see-through display, an application launcher overlaid on the surface of the real-world environment at the gaze target, the application launcher having a world-locked position relative to the real-world environment such that the application launcher does not appear to move relative to the real-world environment when the head-mounted display device moves, the application launcher presenting a plurality of applications selectable for launching within a graphical boundary of the application launcher that occupies a region of the see-through display to allow for the one or more augmentation elements and one or more real-world objects in the real-world environment to be viewed outside of the graphical boundary of the application launcher through the see-through display;
recognize a user selection of one of the plurality of applications; and
replace, via the see-through display, the application launcher with the selected one of the plurality of applications by displaying, in place of the application launcher, a graphical user interface of the selected one of the plurality of applications at the world-locked position overlaid on the surface of the real-world environment where the application launcher was previously displayed such that the graphical user interface of the selected one of the plurality of applications is presented within the graphical boundary of the application launcher.

9. The head-mounted display device of claim 8 , further comprising an input subsystem, wherein the user trigger is detected via the input subsystem.

10. The head-mounted display device of claim 8 , wherein the user trigger comprises a gesture recognized via one or more of the head-mounted display device and a camera remote from the head-mounted display device.

11. The head-mounted display device of claim 8 , further comprising a microphone, and wherein the user trigger comprises a voice command recognized via the microphone.

12. The head-mounted display device of claim 8 , wherein the user trigger comprises a sustained gaze, and wherein the instructions are further executable to recognize the sustained gaze responsive to determining, via the eye-tracking camera, that the gaze target is maintained for a duration that is greater than a trigger threshold.

13. The head-mounted display device of claim 8 , the instructions further executable to, prior to displaying the application launcher, display, via the see-through display, a positional preview of the graphical boundary of the application launcher overlaid on the surface of the real-world environment at the gaze target, the positional preview providing an indication of position, size, and/or shape of the graphical boundary within which the application launcher will be displayed.

14. The head-mounted display device of claim 13 , the instructions further executable to replace, via the see-through display, the positional preview of the graphical boundary of the application launcher with the application launcher presenting the plurality of applications selectable for launching within the graphical boundary of the application launcher responsive to a confirmation input.

15. A head-mounted display device comprising:
an eye-tracking camera;
an input subsystem;
a see-through display;
a logic machine; and
a storage machine holding instructions executable by the logic machine to:
display, via the see-through display, one or more augmentation elements;
recognize, via the eye-tracking camera, a gaze target on a surface in a real-world en

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Lehman Taylor
• Pimmel Kim Pascal
• Beharry Xerxes
• Badh Ranjib
• Zhang Jiashi

Classifications

• G06F2203/0381

  Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G02B27/01

  Head-up displays · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title