System and method for providing user interface tools

Patent metadata

Field	Value
Publication number	US-9766713-B2
Application number	US-201615259952-A
Country	US
Kind code	B2
Filing date	Sep 8, 2016
Priority date	Sep 8, 2015
Publication date	Sep 19, 2017
Grant date	Sep 19, 2017

Abstract

A system includes one or more hardware processors, a head mounted display configured to display a virtual environment to a user, an input device, and a virtual mini-board module. The mini-board module is configured to render the virtual environment for presentation to the user via the HMD, the virtual environment is rendered from a first perspective providing a field of view of the virtual environment to the user, provide a virtual mini-board to the user within the field of view, the virtual mini-board displaying a region of the virtual environment, detect an interaction event performed by the user on the virtual mini-board, identify the first object based on the interaction event performed on the virtual mini-board, and perform the interaction event on the first object within the virtual environment based on the interaction event performed on the virtual mini-board.

First claim

The invention claimed is:

1. A system comprising:
one or more hardware processors;
a head mounted display (HMD) configured to display a virtual environment to a user wearing the HMD;
an input device configured to allow the user to interact with virtual objects presented in the virtual environment; and
a virtual mini-board module, executable by the one or more hardware processors, configured to perform operations comprising:
rendering the virtual environment for presentation to the user via the HMD, the virtual environment being rendered from a first perspective that provides a field of view of the virtual environment to the user;
providing a virtual mini-board to the user within the field of view, the virtual mini-board including one or more virtual mini-objects representing one or more virtual objects presented within a region of the virtual environment, the virtual mini-board is configured to remain located relative to the user within the virtual environment as the user moves within the virtual environment;
detecting an interaction event performed on a first virtual mini-object of the one or more virtual mini-objects by the user on the virtual mini-board, the interaction event being performed by the user using the input device;
identifying a first virtual object of the one or more virtual objects within the virtual environment based on the first virtual mini-object identified by the interaction event performed on the virtual mini-board; and
performing the interaction event on the first object within the virtual environment based on the interaction event performed on the virtual mini-board.

2. The system of claim 1 , wherein providing the virtual mini-board to the user within the field of view further includes orienting the virtual mini-board within a pre-determined length of a location of the user within the virtual environment.

3. The system of claim 2 , wherein providing the virtual mini-board to the user within the field of view further includes moving the virtual mini-board relative to movement of the user within the virtual environment.

4. The system of claim 1 , wherein the input device includes a hand tracking device operated by a first hand of the user, wherein receiving the interaction event includes receiving a grab event involving an object appearing on the virtual mini-board.

5. The system of claim 1 , wherein providing the virtual mini-board further includes rendering the virtual environment onto the virtual mini-board based on a camera position from a second perspective different than the first perspective.

6. The system of claim 1 , wherein providing the virtual mini-board further includes:
identifying a first object appearing within the region of the virtual environment;
creating a first mini-object associated with the first object; and
displaying the first mini-object on the virtual mini-board at a position on the virtual mini-board based on a position of the first object within the region of the virtual environment.

7. The system of claim 1 , wherein the virtual mini-board module is further configured to perform operations comprising:
receiving a scroll operation associated with the region, the scroll operation is initiated by the user scrolling to an edge of the virtual mini-board;
moving the region within the virtual environment in a direction of the edge identified by the scroll operation; and
re-rendering the field of view of the virtual environment based on said moving.

8. A computer-implemented method comprising:
rendering a virtual environment for presentation to the user via a head mounted display (HMD) worn by the user, the virtual environment being rendered from a first perspective that provides a field of view of the virtual environment to the user;
providing a virtual mini-board to the user within the field of view, the virtual mini-board including one or more mini-objects representing one or more virtual objects presented within a region of the virtual environment, the virtual mini-board is configured to remain located relative to the user within the virtual environment as the user moves within the virtual environment;
detecting an interaction event performed on a first virtual mini-object of the one or more virtual mini-objects by the user on the virtual mini-board, the input device is configured to allow the user to interact with virtual objects presented in the virtual environment, the interaction event being performed by the user using the input device;
identifying a first virtual object of the one or more virtual objects based on the first virtual mini-object identified by the interaction event performed on the virtual mini-board; and
performing the interaction event on the first object within the virtual environment based on the interaction event performed on the virtual mini-board.

9. The method of claim 8 , wherein providing the virtual mini-board to the user within the field of view further includes orienting the virtual mini-board within a pre-determined length of a location of the user within the virtual environment.

10. The method of claim 9 , wherein providing the virtual mini-board to the user within the field of view further includes moving the virtual mini-board relative to movement of the user within the virtual environment.

11. The method of claim 8 , wherein the input device includes a hand tracking device operated by a first hand of the user, wherein receiving the interaction event includes receiving a grab event involving an object appearing on the virtual mini-board.

12. The method of claim 8 , wherein providing the virtual mini-board further includes rendering the virtual environment onto the virtual mini-board based on a camera position from a second perspective different than the first perspective.

13. The method of claim 8 , wherein providing the virtual mini-board further includes:
identifying a first object appearing within the region of the virtual environment;
creating a first mini-object associated with the first object; and
displaying the first mini-object on the virtual mini-board at a position on the virtual mini-board based on a position of the first object within the region of the virtual environment.

14. The method of claim 8 , further comprising:
receiving a scroll operation associated with the region, the scroll operation is initiated by the user scrolling to an edge of the virtual mini-board;
moving the region within the virtual environment in a direction of the edge identified by the scroll operation; and
re-rendering the field of view of the virtual environment based on said moving.

15. A non-transitory machine-readable medium storing processor-executable instructions which, when executed by a processor, cause the processor to:
render a virtual environment for presentation to the user via a head mounted display (HMD) worn by the user, the virtual environment being rendered from a first perspective that provides a field of view of the virtual environment to the user;
provide a virtual mini-board to the user within the field of view, the virtual mini-board including one or more mini-objects representing one or more virtual objects presented within a region of the virtual environment, the virtual mini-board is configured to remain located relative to the user within the virtual environment as the user moves within the virtual environment;
detect an interaction event on a first virtual mini-object of the one or more virtual mini-objects performed by the user on the virtual mini-board, the input device is configured to allow the user to interact with virtual objects presented in the virtual environment, the interacti

Assignees

• Unity IPR ApS

Inventors

• West Timoni
• Ebrahimi Amir Pascal

Classifications

• G02B27/01

  Head-up displays · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06K9/00355

  Physics · mapped topic

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T19/00

  Manipulating three-dimensional [3D] models or images for computer graphics · CPC title