Dynamic switching and merging of head, gesture and touch input in virtual reality

Patent metadata

Field	Value
Publication number	US-10606344-B2
Application number	US-201816130040-A
Country	US
Kind code	B2
Filing date	Sep 13, 2018
Priority date	Aug 26, 2015
Publication date	Mar 31, 2020
Grant date	Mar 31, 2020

Abstract

In a system for dynamic switching and merging of head, gesture and touch input in virtual reality, focus may be set on a first virtual in response to a first input implementing one of a number of different input modes. The first object may then be manipulated in the virtual world in response to a second input implementing another input mode. In response to a third input, focus may be shifted from the first object to a second object if, for example, a priority value of the third input is higher than a priority value of the first input. If the priority value of the third input is less than that of the first input, focus may remain on the first object. In response to certain trigger inputs, a display of virtual objects may be shifted between a far field display and a near field display to accommodate a particular mode of interaction with the virtual objects.

First claim

What is claimed is:

1. A method, comprising:
displaying a far field display of a plurality of virtual objects;
detecting a first input;
selecting the far field display including the plurality of virtual objects in response to the first input, the far field display of the plurality of virtual objects appearing outside of a physical reach of a user selecting the far field display;
detecting a second input;
morphing an appearance of each virtual object of the plurality of virtual objects for a near field display of the plurality of virtual objects in response to the second input;
displaying the near field display including the morphed plurality of virtual objects, the near field display of the morphed plurality of virtual objects appearing within the physical reach of the user;
detecting a third input; and
selecting one of the morphed plurality of virtual objects in response to the third input.

2. The method of claim 1 , wherein
detecting the first input includes detecting a far field gaze input or a far field virtual beam input generated by a handheld electronic device, and
detecting the second input includes detecting a hand gesture input or an arm gesture input, triggering a transition from the far field display of the plurality of virtual objects to the near field display of the morphed plurality of virtual objects, and
detecting the third input includes detecting a near field hand gesture or a near field reach gesture, selecting a virtual object from the near field display of morphed virtual objects.

3. The method of claim 2 , further comprising:
detecting a fourth input; and
manipulating the selected virtual object in response to the fourth input.

4. The method of claim 3 , wherein detecting the fourth input includes detecting at least one of a touch and drag input on a touch sensitive surface of the handheld electronic device, a touch on the touch sensitive surface and a movement of the handheld electronic device, or a hand gesture directed to a feature of the selected virtual object.

5. The method of claim 4 , wherein manipulating the selected virtual object in response to the fourth input includes at least one of moving the selected virtual object based on a trace of the touch and drag input, moving the selected virtual object based on the movement of the handheld electronic device, or further morphing the selected virtual object based on the hand gesture.

6. The method of claim 3 , further comprising:
detecting a fifth input;
comparing a priority value of the fifth input to a priority value of the fourth input;
in response to a determination, based on the comparison, that the priority value of the fifth input is greater than the priority value of the fourth input, releasing selection of the currently selected virtual object, and shifting selection to a new virtual object associated with the fifth input; and
in response to a determination, based on the comparison, that the priority value of the fifth input is less than the priority value of the fourth input, maintaining selection of the currently selected virtual object.

7. The method of claim 1 , wherein each input is implemented via one of a plurality of input modes, each of the plurality of input modes having an associated priority value, the respective priority value associated with each of the plurality of input modes being preset or being set by a user in accordance with a user profile.

8. The method of claim 7 , wherein the plurality of input modes includes a head gaze input mode, an eye gaze input mode, a point input mode, a reach input mode, and a gesture input mode.

9. A method, comprising:
displaying a plurality of virtual objects, each of the plurality of virtual objects being selectable and manipulatable in response to a plurality of input modes;
detecting a first input implementing a first input mode of the plurality of input modes;
selecting at least one virtual object of the plurality of virtual objects in response to the first input, the at least one virtual object being displayed in a far field display that appears outside a physical reach of a user selecting the at least one virtual object from the far field display of the plurality of virtual objects;
detecting a second input implementing a second input mode of the plurality of input modes; and
displaying the at least one virtual object in a near field display in response to the second input, the near field display of the at least one virtual object appearing within the physical reach of the user.

10. The method of claim 1 , wherein morphing the appearance of the plurality of virtual objects, and displaying the near field display including the morphed plurality of virtual objects, includes displaying additional features associated with one or more of the plurality of virtual objects, or displaying additional information related to one or more of the plurality of virtual objects, for interaction with the plurality of virtual objects in the near field display.

11. The method of claim 9 , further comprising morphing an appearance of the at least one virtual object for the near field display of the at least one virtual object in response to the second input.

12. The method of claim 11 , wherein morphing the appearance of the at least one virtual object includes decreasing a virtual distance from a user to the at least one virtual object.

13. The method of claim 11 , wherein morphing the appearance of the at least one virtual object, and displaying the at least one virtual object in the near field display, includes displaying additional features associated with the at least one virtual object, or displaying additional information related to the at least one virtual object, for interaction with the at least one virtual object in the near field display.

14. The method of claim 11 , wherein morphing the appearance of the at least one virtual object includes at least one of changing a size of the at least one virtual object, changing an image associated with the at least one virtual object, changing text associated with the at least one virtual object, or changing menu items associated with the at least one virtual object, as the at least one virtual object moves from the far field display to the near field display.

15. The method of claim 9 , further comprising:
detecting a third input implementing a third input mode of the plurality of input modes; and
setting an active focus on a first virtual object in the near field display in response to the third input.

16. The method of claim 15 , further comprising:
detecting a fourth input implementing a fourth input mode of the plurality of input modes;
comparing a priority value of the fourth input mode and a priority value of the third input mode and; and
based on the comparison,
releasing active focus from the first virtual object and shifting active focus to a second virtual object of the plurality of virtual objects in response to the fourth input, or
maintaining selection of the first virtual object in response to the fourth input.

17. The method of claim 15 , further comprising:
detecting a fourth input implementing a fourth input mode of the plurality of input modes; and
manipulating the first virtual object in response to the fourth input.

18. The method of claim 9 , wherein the plurality of input modes includes a head gaze input mode, an eye gaze input mode, a point input mode, a reach input mode, and a gesture input mode.

19. The method of claim 9 , wherein detecting the first input includes detecting a head gaze input, or

Assignees

• Google Llc

Inventors

• Faaborg Alexander James
• Clement Manuel Christian
• Mckenzie Chris

Classifications

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/04847

  Interaction techniques to control parameter settings, e.g. interaction with sliders or dials · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title