Velocity-based controls

Patent metadata

Field	Value
Publication number	US-11423585-B2
Application number	US-202017107784-A
Country	US
Kind code	B2
Filing date	Nov 30, 2020
Priority date	Nov 30, 2020
Publication date	Aug 23, 2022
Grant date	Aug 23, 2022

Abstract

Examples that relate to virtual controls in a mixed reality experience are described. One example provides a method comprising, via a mixed reality display device, displaying mixed reality content including a representation of a virtual control, and receiving sensor data indicating motion of a user digit. The method further comprises, based at least in part on the sensor data, determining a velocity of the user digit, and responsive to determining that the velocity of the user digit relative to a surface corresponding to the virtual control satisfies a velocity-based selection condition, triggering the virtual control.

First claim

The invention claimed is:

1. A method, comprising:
via a mixed reality display device, displaying mixed reality content including a representation of a virtual control;
receiving sensor data indicating motion of a user digit;
responsive to a determination that the virtual control is attached to a physical surface,
(1) based at least in part on the sensor data, determining a velocity of the user digit relative to the surface to which the virtual control is attached, and
(2) responsive to determining that the velocity of the user digit relative to the surface satisfies a velocity-based selection condition, triggering the virtual control; and

responsive to a determination that the virtual control is not attached to the physical surface, using a position-based selection condition and not the velocity-based selection condition to effect the triggering of the virtual control.

2. The method of claim 1 , where the velocity-based selection condition includes the velocity of the user digit relative to the surface decreasing by at least a threshold rate.

3. The method of claim 1 , where the velocity-based selection condition includes an instantaneous velocity of the user digit being a predetermined fraction of an average velocity of the user digit.

4. The method of claim 1 , where determining that the velocity-based selection condition is satisfied is performed while a position of the user digit is within a hit zone at least partially extruded from the representation of the virtual control.

5. The method of claim 1 , where the velocity-based selection condition includes the user digit traversing at least a threshold distance along a normal direction of the representation of the virtual control.

6. The method of claim 1 , where the velocity-based selection condition includes a primary direction of the motion of the user digit being along a normal direction of the representation of the virtual control.

7. The method of claim 1 , further comprising determining an offset between a displayed location of the representation of the virtual control and a perceived location of the representation of the virtual control, such offset being inferred from the motion of the user digit.

8. The method of claim 7 , further comprising adjusting one or more parameters of the velocity-based selection condition based on the offset.

9. The method of claim 7 , further comprising adjusting the displayed location of the representation of the virtual control based on the offset.

10. The method of claim 7 , further comprising, responsive to determining that the user digit has traversed at least a threshold distance away from the representation of the virtual control, releasing the virtual control such that the virtual control is triggerable, and adjusting the threshold distance based on the offset.

11. The method of claim 1 , where the virtual control is a first virtual control attached to the physical surface, the method further comprising:
displaying a representation of a second virtual control not attached to the physical surface;
based at least in part on the sensor data, determining a position of the user digit; and
responsive to determining that the position of the user digit relative to a hit zone corresponding to the second virtual control satisfies the position-based selection condition, triggering the second virtual control.

12. The method of claim 1 , where the surface to which the virtual control is attached is a surface of the virtual control.

13. The method of claim 1 , further comprising using one or more of surface recognition or object recognition to determine that the virtual control is attached to the physical surface or that the virtual control is not attached to the physical surface.

14. A mixed reality display device, comprising:
a display;
a sensor subsystem;
a logic subsystem; and
a storage subsystem holding instructions executable by the logic subsystem to:
via the display, display mixed reality content including a representation of a virtual control;
via the sensor subsystem, receive sensor data indicating motion of a user digit;
responsive to a determination that the virtual control is attached to a physical surface,
(1) based on the sensor data, determine a velocity of the user digit relative to the surface to which the virtual control is attached, and
(2) responsive to determining that the velocity of the user digit relative to the surface satisfies a velocity-based selection condition, trigger the virtual control; and

responsive to a determination that the virtual control is not attached to the physical surface, use a position-based selection condition and not the velocity-based selection condition to effect the triggering of the virtual control.

15. The mixed reality display device of claim 14 , where the velocity-based selection condition includes the velocity of the user digit relative to the surface decreasing below a threshold velocity.

16. The mixed reality display device of claim 14 , where the velocity-based selection condition includes one or more of:
an instantaneous velocity of the user digit being a predetermined fraction of an average velocity of the user digit;
a position of the user digit remaining within a hit zone at least partially extruded from the representation of the virtual control;
the user digit traversing at least a threshold distance along a normal direction of the representation of the virtual control; and
a primary direction of the motion of the user digit being along a normal direction of the representation of the virtual control.

17. The mixed reality display device of claim 14 , further comprising instructions executable to determine an offset between a displayed location of the representation of the virtual control and a perceived location of the representation of the virtual control, such offset being inferred from the motion of the user digit.

18. The mixed reality display device of claim 17 , further comprising instructions executable to adjust one or more parameters of the velocity-based selection condition based on the offset and/or the displayed location of the representation of the virtual control based on the offset.

19. A method, comprising:
via a mixed reality display device, displaying mixed reality content including a representation of a virtual control;
receiving sensor data indicating motion of a user digit;
responsive to a determination that the virtual control is attached to a physical surface,
(1) based on the sensor data, determining a velocity of the user digit relative to the physical surface to which the virtual control is attached, and
(2) responsive to determining that the velocity of the user digit relative to the physical surface satisfies a velocity-based selection condition, triggering the virtual control; and

responsive to a determination that the virtual control is not attached to the physical surface, using a position-based selection condition and not the velocity-based selection condition to effect the triggering of the virtual control.

20. The method of claim 19 , further comprising:
determining an offset between a displayed location of the representation of the virtual control and a perceived location of the representation of the virtual control, such offset being inferred from the motion of the user digit; and
adjusting the displayed location of the representation of the virtual control based on the offset.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Meekhof Casey Leon
• San Kyle Nicholas
• Schwarz Julia

Classifications

• G06F3/04166

  Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving (Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/04883

  for inputting data by handwriting, e.g. gesture or text · CPC title

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title