Discrete and continuous gestures for enabling hand rays

Patent metadata

Field	Value
Publication number	US-11397463-B2
Application number	US-201916297237-A
Country	US
Kind code	B2
Filing date	Mar 8, 2019
Priority date	Jan 12, 2019
Publication date	Jul 26, 2022
Grant date	Jul 26, 2022

Abstract

Systems and methods are provided for selectively enabling or disabling control rays in mixed-reality environments. In some instances, a mixed-reality display device presents a mixed-reality environment to a user which includes one or more holograms. The display device then detects a user gesture input associated with a user control (which may include a part of the user's body) during presentation of the mixed-reality environment. In response to detecting the user gesture, the display device selectively generates and displays a corresponding control ray as a hologram rendered by the display device extending away from the user control within the mixed-reality environment. Gestures may also be detected for selectively disabling control rays so that they are no longer rendered.

First claim

What is claimed is:

1. A system for selectively enabling a display of a control ray in a mixed-reality environment, the system comprising:
one or more processors; and
one or more computer-readable media having stored thereon computer-executable instructions that are executable by the one or more processors to configure the system to perform the following:
present a mixed-reality environment to a user with a mixed-reality display device, the mixed-reality environment including one or more holograms;
obtain a first control ray activation variable associated with a user control, the user control including at least a hand of the user, the first control ray activation variable being based on a function of one or more spatial or angular relationships between individual control members of the user control measuring an openness of the hand of the user;
obtain in a second control ray activation variable associated with the user control, the second control ray activation variable comprising a velocity or acceleration of the user control;
in response to determining that (i) the first control ray activation variable meets or exceeds a predetermined threshold and (ii) the second control ray activation variable does not exceed a second predetermined threshold, selectively enable display of a control ray within the mixed-reality environment, the control ray being rendered as a hologram of a line extending away from the user control within the mixed-reality environment, wherein the control ray is rendered, upon the display thereof being selectively enabled in response to the first control ray activation variable meeting or exceeding the predetermined threshold and the second control ray activation variable not exceeding the second predetermined threshold, with an orientation that is based on an axial alignment of at least a portion of the user control; and
in response to determining that the first control ray activation variable fails to meet or exceed the predetermined threshold or the second control ray activation variable exceeds the second predetermined threshold, selectively disable display of or refrain from displaying the control ray within the mixed-reality environment.

2. The system of claim 1 , wherein the computer-executable instructions are further operable to configure the system to determine an environmental context, wherein the control ray is only selectively enabled upon determining that the environmental context allows for control rays to be enabled and wherein at least one context in the mixed-reality environment allows for control rays to be enabled and at least one context in the mixed-reality environment refrains from allowing control rays to be enabled.

3. The system of claim 1 , wherein the computer-executable instructions are further operable to configure the system to determine a user activity context, wherein the control ray is only selectively enabled upon determining that the user activity context allows for control rays to be enabled.

4. The system of claim 1 , wherein the computer-executable instructions are further operable to configure the system to selectively enable additional user control functionality for interacting with one or more holograms in the mixed-reality environment based on whether the control ray is enabled or disabled.

5. The system of claim 1 , wherein the computer-executable instructions are further operable to configure the system to turn off a display of the control ray within the mixed-reality environment in response to detecting an additional gesture associated with refraining from displaying the control ray.

6. A method for selectively enabling display of a control ray in a mixed-reality environment, the method comprising:
presenting a mixed-reality environment to a user with a mixed-reality display device, the mixed-reality environment including one or more holograms;
detecting a user gesture input associated with a user control during presentation of the mixed-reality environment, the user control including at least a part of a body of the user;
in response to detecting the user gesture input, selectively generating and displaying a control ray as a hologram of a line extending away from the user control within the mixed-reality environment with an orientation that is based on an axial alignment of at least a portion of the user control;
obtaining a control ray activation variable associated with a user control, the user control including at least part of a user's body, the control ray activation variable comprising a dot product between an orientation of the user control and a user gaze vector;
obtaining a second control ray activation variable associated with the user control, the second control ray activation variable comprising a velocity or acceleration of the user control; and after selectively generating and displaying the control ray in response to detecting the user gesture input within the mixed-reality environment, and in response to determining that (i) the dot product between the orientation of the user control and the user gaze vector fails to meet or exceed a predetermined threshold or (ii) the second control ray activation variable exceeds the second predetermined threshold, selectively disabling display of the control ray within the mixed-reality environment.

7. The method of claim 6 , wherein the user control comprises a hand or finger of the user.

8. The method of claim 6 , further comprising: determining an environmental context, wherein the control ray is only selectively enabled upon determining that the environmental context allows for control rays to be enabled and wherein at least one context in the mixed-reality environment allows for control rays to be enabled and at least one context in the mixed-reality environment refrains from allowing control rays to be enabled.

9. The method of claim 6 , further comprising:
determining a user activity context, wherein the control ray is only selectively enabled upon determining that the user activity context allows for control rays to be enabled.

10. The method of claim 6 , further comprising:
selectively enabling additional user control functionality for interacting with one or more holograms in the mixed-reality environment based on whether the control ray is enabled or disabled.

11. The method of claim 6 , further comprising:
turning off a display of the control ray within the mixed-reality environment in response to detecting an additional gesture associated with refraining from displaying the control ray.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Schwarz Julia
• Tang sheng kai
• Meekhof Casey Leon
• Sharkasi Nahil Tawfik
• Stellmach Sophie

Classifications

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title