Methods and apparatus to use predicted actions in virtual reality environments

Patent metadata

Field	Value
Publication number	US-9847079-B2
Application number	US-201615151169-A
Country	US
Kind code	B2
Filing date	May 10, 2016
Priority date	May 10, 2016
Publication date	Dec 19, 2017
Grant date	Dec 19, 2017

Abstract

Methods and apparatus to use predicted actions in VR environments are disclosed. An example method includes predicting a predicted time of a predicted virtual contact of a virtual reality controller with a virtual musical instrument, determining, based on at least one parameter of the predicted virtual contact, a characteristic of a virtual sound the musical instrument would make in response to the virtual contact, and initiating producing the sound before the predicted time of the virtual contact of the controller with the musical instrument.

First claim

What is claimed is:

1. A method comprising:
predicting a predicted time of a predicted virtual contact of a virtual reality controller with a virtual musical instrument;
determining, based on at least one parameter of the predicted virtual contact and a predicted latency, a characteristic of a virtual sound to be produced by the virtual musical instrument in response to the virtual contact; and
initiating producing the virtual sound in response to the predicted latency of the virtual contact of the virtual reality controller with the virtual musical instrument being determined.

2. The method of claim 1 , wherein the parameter for the predicted contact comprises a velocity.

3. The method of claim 1 , wherein the predicting the virtual contact comprises using a determined location and a determined velocity to extrapolate to a predicted future location.

4. The method of claim 3 , further comprising at least one of using a captured image and/or an object tracking to determine the location and/or the velocity.

5. The method of claim 1 , further comprising determining when to initiate producing the virtual sound based on a system computational load.

6. The method of claim 1 , further comprising predicting the at least one parameter of the predicted virtual contact, wherein the at least one parameter comprises at least one of a velocity of impact, a location of impact, a failure to impact, a momentum, a force, a direction of impact, an area of impact, or a missed contact.

7. The method of claim 1 , further comprising, when the contact does not occur, automatically adjusting a position of the virtual musical instrument so the virtual reality controller contacts the virtual musical instrument at another time.

8. The method of claim 1 , further comprising:
determining a characteristic of the contact of the virtual reality controller with the virtual musical instrument; and
predicting a second virtual contact of the virtual reality controller with the virtual musical instrument based on the determining the characteristic of the contact of the virtual reality controller with the virtual musical instrument.

9. The method of claim 1 , further comprising:
determining a gesture of the virtual reality controller; and
adjusting a position parameter associated with the virtual musical instrument in response to the determining the characteristic of the contact of the virtual reality controller on the virtual musical instrument.

10. The method of claim 9 , wherein the position parameter comprises at least one of a location, an angle, and/or a height.

11. The method of claim 1 , further comprising:
determining a gesture of the virtual reality controller; and
removing the virtual musical instrument from a virtual environment in response to the gesture.

12. The method of claim 11 , wherein the second virtual contact comprises at least one of a throw, a toss, a flip, a push, a kick, or a swipe.

13. The method of claim 1 , further comprising:
determining a gesture of the virtual reality controller; and
adding a second virtual musical instrument to a virtual environment in response to the gesture.

14. The method of claim 1 , further comprising:
determining a gesture of the virtual reality controller; and
repositioning the virtual musical instrument in response to the gesture.

15. The method of claim 14 , further comprising applying a position parameter of the repositioned virtual musical instrument to automatically position another virtual object.

16. The method of claim 15 , wherein the another virtual object comprises an assistive device.

17. The method of claim 1 , further comprising rendering the virtual contact of the virtual reality controller with the virtual musical instrument for display inside of a head mounted display device.

18. An apparatus comprising:
a processor; and
a non-transitory machine-readable storage media storing instruments that, when executed, causes the processor to:
predict a predicted time of a predicted virtual contact of a virtual reality controller with a virtual musical instrument;
determine, based on at least one parameter of the predicted virtual contact and a predicted latency, a characteristic of a virtual sound to be produced by the virtual musical instrument in response to the virtual contact; and
initiate producing the virtual sound in response to the predicted latency of the virtual contact of the virtual reality controller with the virtual musical instrument being determined.

19. The apparatus of claim 18 , wherein the instructions, when executed, cause the processor to additionally render the virtual contact of the virtual reality controller with the virtual musical instrument for display inside of a head mounted display device.

20. A non-transitory machine-readable media storing machine-readable instructions that, when executed, cause a machine to at least:
predict a predicted time of a predicted virtual contact of a virtual reality controller with a virtual musical instrument;
determine, based on at least one parameter of the predicted virtual contact and a predicted latency, a characteristic of a virtual sound to be produced by the virtual musical instrument in response to the virtual contact; and
initiate producing of the virtual sound in response to the predicted latency of the virtual contact of the virtual reality controller with the virtual musical instrument being determined.

Assignees

• Google Inc
• Google Llc

Inventors

• Clement Manuel Christian
• Welker Stefan

Classifications

• G10H2220/131

  for abstract geometric visualisation of music, e.g. for interactive editing of musical parameters linked to abstract geometric figures · CPC title

• G10H3/00 Primary

  Instruments in which the tones are generated by electromechanical means · CPC title

• G10H2220/455

  Camera input, e.g. analyzing pictures from a video camera and using the analysis results as control data · CPC title

• G10H1/14

  during execution {(voice controlled instruments G10H5/005)} · CPC title

• G10H2220/401

  3D sensing, i.e. three-dimensional (x, y, z) position or movement sensing · CPC title