Target positioning with gaze tracking

Patent metadata

Field	Value
Publication number	US-9244539-B2
Application number	US-201414149756-A
Country	US
Kind code	B2
Filing date	Jan 7, 2014
Priority date	Jan 7, 2014
Publication date	Jan 26, 2016
Grant date	Jan 26, 2016

Abstract

Embodiments that relate to positioning a target indicator via a display system are disclosed. For example, one disclosed embodiment provides a method for positioning a target indicator using gaze tracking data having a coarse accuracy from a gaze tracking system of a computing device. Head pose data having a fine accuracy greater than the coarse accuracy is received from a head tracking system. Using the gaze tracking data, an approximate user gaze region within a display region is determined, and the target indicator is displayed at an initial location within the approximate user gaze region. A reposition input from the user is received. In response, subsequently received head pose data is used to calculate an adjusted location for the target indicator. The target indicator is then displayed at the adjusted location.

First claim

The invention claimed is:

1. A head-mounted computing device for positioning a target indicator displayed via a display system, the head-mounted computing device operatively connected to the display system, the head-mounted computing device comprising:
a gaze tracking system for generating gaze tracking data having a coarse accuracy from a user wearing the head-mounted computing device;
a head tracking system for generating head pose data from the user, wherein the head pose data has a fine accuracy that is greater than the coarse accuracy of the gaze tracking data; and
a positioning module executed by a processor of the head-mounted computing device, the positioning module configured to:
using the gaze tracking data, determine an approximate user gaze region within a display region generated by the display system;
control the display system to display the target indicator at an initial location within the approximate user gaze region;
receive a reposition input from the user;
in response to receiving the reposition input, use subsequently received head pose data to calculate an adjusted location for the target indicator in the display region; and
control the display system to display the target indicator at the adjusted location in the display region.

2. The head-mounted computing device of claim 1 , wherein the positioning module is further configured to:
after the target indicator is displayed at the adjusted location, receive a selection user input; and
in response to receiving the selection user input, select a selectable target located at the adjusted location.

3. The head-mounted computing device of claim 1 , wherein the reposition input from the user is received from an input device selected from the group consisting of a hand-held mechanically actuatable stylus, a depth camera, a ring, a microphone, and the gaze tracking system.

4. The head-mounted computing device of claim 1 , wherein the target indicator is not displayed until the reposition input is received.

5. The head-mounted computing device of claim 1 , wherein the positioning module is further configured to, in response to receiving the reposition input, disable the gaze tracking system.

6. The head-mounted computing device of claim 1 , wherein the display system comprises a near-eye display system integrated in the head-mounted computing device.

7. The head-mounted computing device of claim 1 , wherein the display system comprises a screen in a physical environment and separate from the head-mounted computing device, the head-mounted computing device further comprises an optical sensor for capturing images of the physical environment including the screen, and the positioning module is further configured to:
use the images to establish a spatial relationship between the head-mounted computing device and the screen; and
use the spatial relationship in combination with the gaze tracking data to determine the approximate user gaze region within the display region of the screen.

8. The head-mounted computing device of claim 7 , wherein the positioning module is further configured to receive a screen identifier from the screen.

9. The head-mounted computing device of claim 1 , wherein the positioning module is further configured to:
receive secondary tracking data; and
in response to receiving the reposition input, use subsequently received secondary tracking data to calculate the adjusted location for the target indicator.

10. A method for positioning a target indicator displayed via a display system, the method comprising:
receiving gaze tracking data having a coarse accuracy from a gaze tracking system of a computing device;
receiving head pose data having a fine accuracy that is greater than the coarse accuracy of the gaze tracking data, the head pose data received from a head tracking system of the computing device;
using the gaze tracking data, determining an approximate user gaze region within a display region generated by the display system;
controlling the display system to display the target indicator at an initial location within the approximate user gaze region;
receiving a reposition input from the user;
in response to receiving the reposition input, using subsequently received head pose data to calculate an adjusted location for the target indicator in the display region; and
controlling the display system to display the target indicator at the adjusted location in the display region.

11. The method of claim 10 , further comprising:
after the target indicator is displayed at the adjusted location, receiving a selection user input; and
in response to receiving the selection user input, selecting a selectable target located at the adjusted location.

12. The method of claim 10 , wherein the reposition input from the user is received from an input device selected from the group consisting of a hand-held mechanically actuatable stylus, a depth camera, a ring, a microphone, and the gaze tracking system.

13. The method of claim 10 , further comprising not displaying the target indicator until the reposition input is received.

14. The method of claim 10 , further comprising disabling the gaze tracking system in response to receiving the reposition input.

15. The method of claim 10 , wherein the display system comprises a screen integrated in the computing device.

16. The method of claim 10 , wherein the display system comprises a screen in a physical environment and separate from the computing device, the computing device further comprises an optical sensor for capturing images of the physical environment including the screen, and the method further comprising:
using the images to establish a spatial relationship between the computing device and the screen; and
using the spatial relationship in combination with the gaze tracking data to determine the approximate user gaze region within the display region of the screen.

17. The method of claim 16 , further comprising receiving a screen identifier from the screen.

18. The method of claim 10 , further comprising:
receiving secondary tracking data; and
in response to receiving the reposition input, using the secondary tracking data to calculate the adjusted location for the target indicator.

19. A head-mounted display device for positioning a target indicator displayed via a display system, the head-mounted display device operatively connected to a computing device, the head-mounted display device comprising:
a gaze tracking system for generating gaze tracking data having a coarse accuracy from a user wearing the head-mounted computing device;
a head tracking system for generating head pose data from the user, wherein the head pose data has a fine accuracy that is greater than the coarse accuracy of the gaze tracking data; and
a positioning module executed by a processor of the computing device, the positioning module configured to:
using the gaze tracking data, determine an approximate user gaze region within a display region generated by the display system;
receive a reposition input from the user;
in response to receiving the reposition input, control the display system to display the target indicator at an initial location within the approximate user gaze region;
in response to receiving the reposition input, use subsequently received head pose data to calculate an adjusted location for the target indicator in the display region;
control the display system to display the target indicator at the adjusted location in the display region;
after the target indicator is displ

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Venable Morgan Kolya
• Kerr Bernard James
• Wilairat Weerapan

Classifications

• A61B3/113

  for determining or recording eye movement · CPC title

• G06F3/038

  Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry · CPC title

• G06F2201/805

  Real-time · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/0346 Primary

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title