Facilitate user manipulation of a virtual reality environment view using a computing device with touch sensitive surface

Patent metadata

Field	Value
Publication number	US-10627908-B2
Application number	US-201514871611-A
Country	US
Kind code	B2
Filing date	Sep 30, 2015
Priority date	Mar 27, 2015
Publication date	Apr 21, 2020
Grant date	Apr 21, 2020

Abstract

A system and method for controlling a view of a virtual reality (VR) environment via a computing device with a touch sensitive surface are disclosed. In some examples, a user may be enabled to augment the view of the VR environment by providing finger gestures to the touch sensitive surface. In one example, the user is enabled to call up a menu in the view of the VR environment. In one example, the user is enabled to switch the view of the VR environment displayed on a device associated with another user to a new location within the VR environment. In some examples, the user may be enabled to use the computing device to control a virtual camera within the VR environment and have various information regarding one or more aspects of the virtual camera displayed in the view of the VR environment presented to the user.

First claim

What is claimed is:

1. A computer-implemented method for facilitating a user to manipulate a view of a virtual reality environment, the method comprising:
receiving, by a server, a sensor input indicating movement of a mobile display device associated with a user, the sensor input corresponding to a navigation command for controlling navigation within the virtual reality environment;
determining, by the server, based on the sensor input, a first location within the virtual reality environment where a first view of the virtual reality environment is taken;
rendering, by the server, the first view of the virtual reality environment for presentation in a head mount device (HMD);
receiving over a network, by the server, from a portable device having a touch sensitive surface, the portable device being associated with the user, a first user input by a first user, wherein the first user input is provided by the user to change the first view of the virtual reality environment, and the first user input includes a finger swiping gesture;
mapping, by the server, the first user input to a first command for manipulating the view of the virtual reality environment;
determining a position for display a menu of selectable items in the first view based on locations of one or more virtual objects or characters in the first view;
based on the first view and the first command, determining, by the server, a second view of the virtual reality environment, wherein the second view includes the menu of selectable items at the determined position; and
rendering, by the server, one or more images representing the second view for presentation in the HMD.

2. The method of claim 1 , wherein the determination of the second view comprises:
determining a context of the first view; and
determining the second view based on the context of the first view.

3. The method of claim 2 , wherein the context of the first view includes the first location, a size of a field of the first view, a subject matter of the first view, one or more virtual objects in the first view, and/or one or more virtual characters in the first view.

4. The method of claim 1 , wherein each of the selectable items of the menu correspond to one or more control commands.

5. The method of claim 4 , wherein the determination of the second view includes determining a menu type associated with the menu, determining the selectable items for inclusion in the menu, determining a layout of the menu for presentation in the first view, or determining a position within the first view at which the menu is presented.

6. The method of claim 1 , further comprising transmitting the one or more images to a computing device for presentation.

7. The method of claim 1 , further comprising transmitting the one or more images to a head-mounted display for presentation.

8. The method of claim 1 , further comprising transmitting the one or more images to a head-mounted display associated with a second user for presentation.

9. The method of claim 1 , wherein the first user input includes a finger gesture provided by the first user via the touch sensitive surface of the portable device.

10. A server for facilitating a user to manipulate a view of a virtual reality environment, the server comprising:
a processor configured to:
receive a sensor input indicating movement of a mobile display device associated with a user, the sensor input corresponding to a navigation command for controlling navigation within the virtual reality environment;
determine, based on the sensor input, a first view location within the virtual reality environment where a first view of the virtual reality environment is taken;
render the first view of the virtual reality environment for presentation in a head mount device (HMD);
receive over a network, from a portable device having a touch sensitive surface, the portable device being associated with the user, a first user input by a first user, wherein the first user input is provided by the user to change the first view of the virtual reality environment and the first user input includes a finger swiping gesture;
map the first user input to a first command for manipulating the view of the virtual reality environment;
determine a position for display a menu of selectable items in the first view based on locations of one or more virtual objects or characters in the first view;
based on the first view and the first command, determine a second view of the virtual reality environment, wherein the second view includes the menu of selectable items at the determined position; and
render one or more images representing the second view for presentation in a head mount device (HMD).

11. The server of claim 10 , wherein the determination of the second view comprises:
determining a context of the first view; and
determining the second view based on the context of the first view.

12. The server of claim 11 , wherein the context of the first view includes the first location, a size of a field of the first view, a subject matter of the first view, one or more virtual objects in the first view, and/or one or more virtual characters in the first view.

13. The server of claim 10 , wherein each of the selectable items of the menu corresponds to one or more control commands.

14. The server of claim 13 , wherein the determination of the second view includes determining a menu type associated with the menu, determining the selectable items for inclusion in the menu, determining a layout of the menu for presentation in the first view, or determining a position within the first view at which the menu is presented.

15. The server of claim 10 , wherein the processor is further configured to transmit the one or more images to a computing device for presentation.

16. The server of claim 10 , wherein the processor is further configured to transmit the one or more images to a head-mounted display for presentation.

17. The server of claim 10 , wherein the processor is further configured to transmit the one or more images to a head-mounted display associated with a second user for presentation.

18. The server of claim 10 , wherein the first user input includes a finger gesture provided by the first user via the touch sensitive surface of the portable device.

Assignees

• Lucasfilm Entertainment Co Ltd

Inventors

• Johnston Darby
• Wakelin Ian

Classifications

• H04N13/398

  Synchronisation thereof; Control thereof · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G02B27/01

  Head-up displays · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title