Virtual keyboard engagement

Patent metadata

Field	Value
Publication number	US-11029845-B2
Application number	US-201916509267-A
Country	US
Kind code	B2
Filing date	Jul 11, 2019
Priority date	Jul 11, 2019
Publication date	Jun 8, 2021
Grant date	Jun 8, 2021

Abstract

The discussion relates to virtual keyboard engagement. One example can define key volumes relating to keys of a virtual keyboard and detect finger movement of a user through individual key volumes. The example can detect parameter changes associated with detected finger movement through individual key volumes and build potential key sequences from detected parameter changes.

First claim

The invention claimed is:

1. A device, comprising:
a display component configured to be positioned on a head of a user in front of the user's eyes and configured to generate virtual content that appears to the user in a scene and that includes virtual keys;
sensors configured to monitor frustums defined by individual virtual keys that extend from the user's eyes through the individual virtual keys relative to the user; and,
a processor configured to assign first probabilities to user movement through individual frustums that is parallel to the virtual content and to assign second higher probabilities to user movement through the individual frustums that is toward or away from the user and to predict an intended user action relative to the virtual content based at least in part on the first probabilities and the second probabilities.

2. The device of claim 1 , wherein the sensors are configured to detect parameters associated with the movement of a user extremity through the frustums.

3. The device of claim 2 , wherein the parameters relate to velocity and direction relative to the individual frustums.

4. The device of claim 3 , wherein the virtual content is associated with the individual virtual keys associated with characters, and wherein the processor is configured to compare a sequence of characters associated with the movement to a dictionary of known words to predict the intended user action as spelling an individual word from the dictionary.

5. The device of claim 4 , wherein the processor is configured to perform the intended user action.

6. The device of claim 5 , wherein the processor is configured to perform the intended user action by auto-populating the individual word into a dialog box or providing the individual word as an auto-suggestion for the dialog box.

7. The device of claim 6 , wherein the processor is configured to perform the auto-populating when a confidence associated with the individual word exceeds a threshold.

8. A device-implemented method, comprising:
generating a virtual keyboard with head mounted optics positioned in front of eyes of a user;
detecting a path of a user extremity in 3 D space proximate to a perceived position of the virtual keyboard;
assigning probabilities to individual keys of the virtual keyboard when the path passes through a frustum defined at least in part by the individual keys;
updating the probabilities relative to an individual key to a higher probability when the path relative to the individual key includes movement either toward or away from the user in the frustum;
predicting an intended user action based at least in part on the probabilities and the updated probabilities; and,
automatically taking an action on behalf of the user based at least in part upon the predicted intended user action.

9. The device-implemented method of claim 8 , wherein detecting a path of a user extremity comprises detecting the path of a finger, multiple fingers, a hand, fingers of both a left and right hand, and/or an object held in the hand.

10. The device-implemented method of claim 8 , wherein assigning probabilities to individual keys of the virtual keyboard comprises assigning a first probability to the individual key of the keyboard when the path passes relative to the individual key under first parameters and assigning a second higher probability to another individual key when the path passes relative to the another individual key under second parameters.

11. The device-implemented method of claim 10 , wherein taking an action comprises populating a dialog box with a word selected based at least in part upon the at least one of the first probability, the second higher probability, and/or the updated probabilities.

12. A device-implemented method, comprising:
generating virtual content with a device positioned on a head of a user, the virtual content including virtual keys and appearing to the user in a volume beyond the device;
defining virtual content frustums extending from the device and around a periphery of individual virtual keys,
detecting finger movement of a user through individual virtual content frustums;
detecting parameter changes associated with detected finger movement through the individual virtual content frustums, the parameter changes distinguishing finger movement through the virtual content frustums that is generally parallel to the virtual content from finger movement that is toward or away from the user relative to individual frustums; and,
building potential user selections of individual virtual keys from detected parameter changes at least in part by weighting the finger movement toward or away from the user higher than finger movement parallel to the virtual content.

13. The device-implemented method of claim 12 , wherein generating virtual content comprises generating a virtual keyboard and the defining virtual content frustums are defined at least in part by eyes of the user and the periphery of the individual virtual keys.

14. The device-implemented method of claim 13 , wherein detecting parameter changes comprises detecting at least one of changes in direction and changes in velocity.

15. The device-implemented method of claim 14 , wherein detecting parameter changes in direction comprises at least one of: detecting changes in direction parallel to a key surface and detecting changes in a direction perpendicular to the key surface.

16. The device-implemented method of claim 15 , wherein the detecting parameter changes further comprises assigning higher probabilities to individual virtual keys in potential key sequences associated with parameter changes.

17. The device-implemented method of claim 16 , further comprising comparing the potential key sequences to known words.

18. The device-implemented method of claim 16 , further comprising comparing the potential key sequences to known user commands.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Becker Christopher M
• Eldirghami Nazeeh A
• Barnes Kevin W
• Schwarz Julia
• Carter Eric

Classifications

• G06F3/04886 Primary

  by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title