Methods and apparatus to use predicted actions in virtual reality environments

Patent metadata

Field	Value
Publication number	US-10573288-B2
Application number	US-201715834540-A
Country	US
Kind code	B2
Filing date	Dec 7, 2017
Priority date	May 10, 2016
Publication date	Feb 25, 2020
Grant date	Feb 25, 2020

Abstract

Methods and apparatus to use predicted actions in VR environments are disclosed. An example method includes predicting a predicted time of a predicted virtual contact of a virtual reality controller with a virtual object, determining, based on at least one parameter of the predicted virtual contact, a characteristic of a virtual output the object would make in response to the virtual contact, and initiating producing the virtual output before the predicted time of the virtual contact of the controller with the virtual object.

First claim

What is claimed is:

1. A method comprising:
predicting a predicted time of a predicted virtual contact of a virtual reality controller with a virtual object within a virtual environment displayed by head-mounted device;
determining, based on at least one parameter of the predicted virtual contact and a predicted latency, a characteristic of a virtual output to be produced by the virtual object in response to the virtual contact; and
initiating producing the virtual output in response to the predicted latency of the virtual contact of the virtual reality controller with the virtual object being determined.

2. The method of claim 1 , wherein the virtual object is at least one of a musical instrument, a document, a household item, a door knob, or a table.

3. The method of claim 1 , wherein the virtual output is at least one of sound, light, color of light, color saturation, or acoustic shape of a sound.

4. The method of claim 1 , wherein the predicting the virtual contact is predicted using a determined location and a determined velocity to extrapolate to a predicted future location.

5. The method of claim 1 , further comprising predicting the at least one parameter of the predicted virtual contact, wherein the at least one parameter comprises at least one of a velocity of impact, a location of impact, a failure to impact, a momentum, a force, a direction of impact, an area of impact, or a missed contact.

6. The method of claim 1 , further comprising, when the contact does not occur, automatically adjusting a position of the virtual object so the virtual reality controller contacts the virtual object at another time.

7. The method of claim 1 , further comprising:
determining a characteristic of the virtual contact of the virtual reality controller with the virtual object, the virtual contact being a first virtual contact; and
predicting a second virtual contact of the virtual reality controller with the virtual object based on the determining the characteristic of the first virtual contact of the virtual reality controller with the virtual object.

8. The method of claim 7 , further comprising:
determining a gesture of the virtual reality controller; and
adjusting a position parameter associated with the virtual object in response to the determining the characteristic of the virtual contact of the virtual reality controller on the virtual object.

9. The method of claim 8 , wherein the position parameter comprises at least one of a location, an angle, or a height.

10. The method of claim 8 , wherein the gesture includes at least one of a throw, a toss, a flip, a push, a kick, or a swipe.

11. The method of claim 1 , further comprising: determining a gesture of the virtual reality controller; and repositioning the virtual object in response to the gesture.

12. The method of claim 11 , further comprising applying a position parameter of the repositioned virtual object to automatically position another virtual object.

13. An apparatus comprising:
a processor; and a non-transitory machine-readable storage media storing instruments that, when executed, causes the processor to:
determine a current location, a current direction and a current velocity of a virtual reality controller with respect to a virtual object within a virtual environment displayed by a head-mounted device;
predict a predicted time of a predicted virtual contact of the virtual reality controller with the virtual object; initiate producing a virtual output based on the predicted time of the virtual contact of the virtual reality controller with the virtual object; and

determine a predicted future location based on at least the current location, the current direction and the current velocity.

14. The apparatus of claim 13 , wherein the virtual object is at least one of a musical instrument, a document, a household item, a door knob, or a table.

15. The apparatus of claim 13 , wherein the virtual output is at least one of sound, light, color of light, color saturation, or acoustic shape of a sound.

16. The apparatus of claim 13 , further comprising tracking a predicted latency from when the virtual output is initiated and when the object output is started to be rendered.

17. The apparatus of claim 16 , wherein the predicted latency is determined from at least one of an average, a windowed average, a moving average, or an exponential average.

18. A non-transitory machine-readable media storing machine-readable instructions that, when executed, cause a machine to at least:
predict a predicted time of a predicted virtual contact of a virtual reality controller with a virtual object within a virtual environment displayed by a head-mounted device;
determine, based on at least one parameter of the predicted virtual contact and a predicted latency, a characteristic of a virtual output to be produced by the virtual object in response to the virtual contact; and
initiate producing the virtual output in response to the predicted latency of the virtual contact of the virtual reality controller with the virtual object being determined.

19. The non-transitory media of claim 18 , wherein the predicted virtual contact is predicted using the at least one parameter to determine a predicted future location.

20. The non-transitory media of claim 19 , wherein the at least one parameter comprises at least one of a velocity of impact or a location of impact.

Assignees

• Google Llc

Inventors

• Clement Manuel Christian
• Welker Stefan

Classifications

• G10H2220/455

  Camera input, e.g. analyzing pictures from a video camera and using the analysis results as control data · CPC title

• G10H2220/401

  3D sensing, i.e. three-dimensional (x, y, z) position or movement sensing · CPC title

• G10H2220/131

  for abstract geometric visualisation of music, e.g. for interactive editing of musical parameters linked to abstract geometric figures · CPC title

• G10H7/008

  Means for controlling the transition from one tone waveform to another · CPC title

• G10H2220/201

  for movement interpretation, i.e. capturing and recognizing a gesture or a specific kind of movement, e.g. to control a musical instrument · CPC title