Virtual affordance display at virtual target

Patent metadata

Field	Value
Publication number	US-9983684-B2
Application number	US-201615341957-A
Country	US
Kind code	B2
Filing date	Nov 2, 2016
Priority date	Nov 2, 2016
Publication date	May 29, 2018
Grant date	May 29, 2018

Abstract

Methods and devices for displaying a virtual affordance with a virtual target are disclosed. In one example, the virtual target is displayed to a user via a display device. The user's point of gaze is determined to be at a gaze location within a target zone including the virtual target. The user's hand is determined to be at a hand location within a designated tracking volume. Based on at least determining that the user's gaze is at the gaze location and the user's hand is at the hand location, the virtual affordance is displayed at a landing location corresponding to the virtual target, where the landing location is independent of both the gaze location and the user's hand location. Movement of the user's hand is tracked and the virtual affordance is modified in response to the movement.

First claim

The invention claimed is:

1. A method for displaying a virtual affordance with a virtual target, the method comprising:
displaying the virtual target to a user via a display device;
determining that a point of gaze of the user is at a gaze location that is within a target zone including the virtual target;
determining that a hand of the user is at a hand location within a designated tracking volume;
based on at least determining that the point of gaze of the user is at the gaze location and determining that the hand of the user is at the hand location, displaying the virtual affordance at a landing location corresponding to the virtual target via the display device, wherein the landing location is independent of both the gaze location within the target zone and the hand location within the designated tracking volume;
tracking movement of the hand; and
modifying the virtual affordance in response to the movement of the hand.

2. The method of claim 1 , further comprising, based on determining that the point of gaze of the user is at the gaze location within the target zone, displaying an indication of the target zone.

3. The method of claim 1 , wherein displaying the virtual affordance at the landing location via the display device is further based on determining that the hand of the user has moved less than a predetermined amount over a predetermined time period.

4. The method of claim 1 , wherein the landing location comprises a location at which the virtual affordance at least partially overlaps the virtual target.

5. The method of claim 1 , wherein the landing location is spaced from a surface of the virtual target.

6. The method of claim 1 , wherein the display device comprises an at least partially see-through display, and the virtual affordance and the virtual target are displayed to appear in a real world physical environment.

7. The method of claim 1 , further comprising:
using data captured by a sensor, tracking the user's hand moving in a direction to a first adjusted location;
in response to tracking the user's hand moving to the first adjusted location, displaying the virtual affordance moving to a boundary of the target zone;
using additional data captured by the sensor, continuing to track the user's hand moving in the direction from the first adjusted location to a second adjusted location; and
in response to continuing to track the user's hand moving in the direction to the second adjusted location, either (1) continuing to display the virtual affordance at the boundary of the target zone or (2) displaying the virtual affordance at the landing location.

8. The method of claim 1 , wherein modifying the display of the virtual affordance in response to the movement of the user's hand further comprises constraining movement of the virtual affordance along an axis of the virtual target.

9. The method of claim 1 , further comprising:
determining that the point of gaze of the user has exited the target zone; and
in response to determining that the point of gaze of the user has exited the target zone, ceasing to display the virtual affordance.

10. The method of claim 1 , wherein determining that the hand of the user is at the hand location further comprises capturing image data of the hand using an image sensor, the method further comprising:
operating the image sensor at a first capture rate when the hand is outside the designated tracking volume; and
operating the image sensor at a second capture rate greater than the first capture rate when the hand of the user is within the designated tracking volume.

11. The method of claim 1 , wherein the hand is a right hand of the user, the hand location is a right hand location, the landing location is a right hand landing location, and the virtual affordance is a right hand virtual affordance, the method further comprising:
determining that a left hand of the user is at a left hand location within the designated tracking volume;
based on at least determining that the point of gaze of the user is at the gaze location and determining that the left hand of the user is at the left hand location, displaying a left hand virtual affordance at a left hand landing location via the display device, wherein the left hand landing location is different from the right hand landing location and is independent of both the gaze location within the target zone and the left hand location within the designated tracking volume;
tracking movement of the user's left hand; and
modifying the display of the left hand virtual affordance in response to the movement of the user's left hand.

12. A computing device for displaying a virtual affordance with a virtual target, the computing device comprising:
a processor; and
a memory holding instructions executable by the processor to:
display the virtual target to a user via a display device;
determine that a point of gaze of the user is at a gaze location that is within a target zone including the virtual target;
determine that a hand of the user is at a hand location within a designated tracking volume;
based on at least determining that the point of gaze of the user is at the gaze location and determining that the hand of the user is at the hand location, display the virtual affordance at the landing location via the display device, wherein the landing location is independent of both the gaze location within the target zone and the hand location within the designated tracking volume;
track movement of the hand; and
modify the virtual affordance in response to the movement of the hand.

13. The computing device of claim 12 , wherein the instructions are further executable to establish the landing location at a location at which the virtual affordance at least partially overlaps the virtual target.

14. The computing device of claim 12 , wherein the instructions are further executable to display an indication of the target zone based on determining that the point of gaze of the user is at the gaze location within the target zone.

15. The computing device of claim 12 , wherein displaying the virtual affordance at the landing location via the display device is further based on determining that the hand of the user has moved less than a predetermined amount over a predetermined time period.

16. The computing device of claim 12 , wherein the instructions are further executable to:
using data captured by a sensor, track the user's hand moving in a direction to a first adjusted location;
in response to tracking the user's hand moving to the first adjusted location, display the virtual affordance moving to a boundary of the target zone;
using additional data captured by the sensor, continue to track the user's hand moving in the direction from the first adjusted location to a second adjusted location; and
in response to continuing to track the user's hand moving in the direction to the second adjusted location, either (1) continue to display the virtual affordance at the boundary of the target zone or (2) display the virtual affordance at the landing location.

17. The computing device of claim 12 , wherein modifying the display of the virtual affordance in response to the movement of the user's hand further comprises constraining movement of the virtual affordance along an axis of the virtual target.

18. The computing device of claim 12 , wherein determining that the hand of the user is at the hand location further comprises capturing image data of the hand using an image sensor, and the instructions are further executable to:
operate the image sensor at a fi

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Wang Jia
• Sheri Yasaman
• Schwarz Julia
• Calabrese David J
• Witriol Daniel B

Classifications

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title