Facilitate user manipulation of a virtual reality environment view using a computing device with a touch sensitive surface

Patent metadata

Field	Value
Publication number	US-11099654-B2
Application number	US-202016852233-A
Country	US
Kind code	B2
Filing date	Apr 17, 2020
Priority date	Mar 27, 2015
Publication date	Aug 24, 2021
Grant date	Aug 24, 2021

Abstract

A system and method for controlling a view of a virtual reality (VR) environment via a computing device with a touch sensitive surface are disclosed. In some examples, a user may be enabled to augment the view of the VR environment by providing finger gestures to the touch sensitive surface. In one example, the user is enabled to call up a menu in the view of the VR environment. In one example, the user is enabled to switch the view of the VR environment displayed on a device associated with another user to a new location within the VR environment. In some examples, the user may be enabled to use the computing device to control a virtual camera within the VR environment and have various information regarding one or more aspects of the virtual camera displayed in the view of the VR environment presented to the user.

First claim

What is claimed is:

1. A computer-implemented method for facilitating a user to manipulate a view of a virtual environment, the method comprising:
receiving, by a server, a sensor input indicating movement of a mobile display device associated with a user, the sensor input corresponding to a navigation command for controlling navigation within the virtual environment;
determining, by the server, based on the sensor input, a first location within the virtual environment where a first view of a virtual reality environment is taken;
determining, by the server, a second location within the virtual environment associated with a second user where a second view of a virtual reality environment is taken;
rendering, by the server, one or more images representing the first view of the virtual environment corresponding to the first location;
receiving a first user input over a network, by the server, from a portable device having a touch sensitive surface, the portable device being associated with the user, wherein the first user input is provided by the user to manipulate the first view of the virtual environment, and the first user input includes one or more first finger gestures;
mapping, by the server, the first user input to a first command for manipulating the first view of the virtual environment;
based on the first view and the first command, determining, by the server, a third view of the virtual environment, wherein the third view is taken at a third location in the virtual environment, wherein the third location is distinct from the first location, and the second location;
receiving a second user input over the network, by the server, from the portable device having the touch sensitive surface the second user input includes one or more second finger gestures that are different from the one or more first finger gestures; and
rendering, by the server, one or more images representing the second view of the virtual environment, wherein the one or more images representing the first view of the virtual environment changes to the one or more images representing the third view of the virtual environment instantly in response to the first user input.

2. The computer-implemented method of claim 1 , wherein the sensor input indicates user position change within an area.

3. The computer-implemented method of claim 1 , wherein the determining the first location further comprises determining at least one of a size of a field of view, a zoom ratio of the first view, or a lens type for presenting the first view.

4. The computer-implemented method of claim 1 , wherein the one or more first finger gestures includes at least one of a multi-finger touch, a reverse pinch, or a rotational finger movement.

5. The computer-implemented method of claim 1 , further comprising transmitting the one or more images of the first view to a head-mounted display for presentation.

6. The computer-implemented method of claim 1 , further comprising transmitting the one or more images of the second view to a head-mounted display for presentation.

7. The computer-implemented method of claim 1 , wherein the one or more second finger gestures comprises a press gesture on the touch sensitive surface.

8. The computer-implemented method of claim 1 , wherein the server establishes a secure connection with each network device using a protocol comprising at least one of a Session Traversal Utilities for NAT (STUN), a Traversal Using Relay NAT (TURN), an Interactive Connectivity Establishment (ICE), a combination thereof, or any other NAT traversal protocol.

9. A server for facilitating a user to manipulate a view of virtual environment, the server comprising:
one or more displays;
one or more processors connected to the one or more displays, wherein the one or more processors are configured to:
receive, by a server, a sensor input indicating movement of a mobile display device associated with a user, the sensor input corresponding to a navigation command for controlling navigation within the virtual environment;
determine, by the server, based on the sensor input, a first location within the virtual environment where a first view of a virtual reality environment is taken;
determine, by the server, a second location within the virtual environment associated with a second user where a second view of a virtual reality environment is taken;
render, by the server, one or more images representing the first view of the virtual environment corresponding to the first location;
receive a first user input over a network, by the server, from a portable device having a touch sensitive surface, the portable device being associated with the user, wherein the first user input is provided by the user to manipulate the first view of the virtual environment, and the first user input includes one or more first finger gestures;
map, by the server, the first user input to a first command for manipulating the first view of the virtual environment;
based on the first view and the first command, determine, by the server, a third view of the virtual environment, wherein the third view is taken at a third location in the virtual environment, wherein the third location is distinct from the first location and the second location;
receive a second user input over the network, by the server, from the portable device having the touch sensitive surface the second user input includes one or more second finger gestures that are different from the one or more first finger gestures; and
render, by the server, one or more images representing the second view of the virtual environment, wherein the one or more images representing the first view of the virtual environment changes to the one or more images representing the third view of the virtual environment instantly in response to the first user input.

10. The server of claim 9 , wherein the sensor input indicates user position change within an area.

11. The server of claim 9 , wherein the determining the first location further comprises determining at least one of a size of a field of view, a zoom ratio of the first view, or a lens type for presenting the first view.

12. The server of claim 9 , wherein the one or more first finger gestures includes at least one of a multi-finger touch, a reverse pinch, or a rotational finger movement.

13. The server of claim 9 , wherein the one or more processors are further configured to transmit the one or more images of the first view to a head-mounted display for presentation.

14. The server of claim 9 , wherein the one or more processors are further configured to transmit the one or more images of the second view to a head-mounted display for presentation.

15. The server of claim 9 , wherein the one or more second finger gestures comprises a press gesture on the touch sensitive surface.

16. A non-transitory computer-readable medium storing a plurality of instructions executable by one or more processors of a computing device of a server, the plurality of instructions comprising:
instructions that cause the one or more processors to receive a sensor input indicating movement of a mobile display device associated with a user, the sensor input corresponding to a navigation command for controlling navigation within a virtual environment;
instructions that cause the one or more processors to determine based on the sensor input, a first location within the virtual environment where a first view of a virtual reality environment is taken;
instructions that cause the one or more processors to determine, by the server, a second location within the virtual environment associated with a

Assignees

• Lucasfilm Entertainment Co Ltd

Inventors

• Johnston Darby
• Wakelin Ian

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• H04N23/695

  Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects · CPC title

• H04N23/631

  Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters · CPC title

• H04N23/60

  Control of cameras or camera modules · CPC title

• H04N23/62

  Control of parameters via user interfaces · CPC title