Interaction engine for creating a realistic experience in virtual reality/augmented reality environments

Patent metadata

Field	Value
Publication number	US-10866632-B2
Application number	US-201916588902-A
Country	US
Kind code	B2
Filing date	Sep 30, 2019
Priority date	Feb 13, 2015
Publication date	Dec 15, 2020
Grant date	Dec 15, 2020

Abstract

The technology disclosed relates to a method of realistic simulation of real world interactions as virtual interactions between a control object sensed acting in a three-dimensional (3D) sensory space and the virtual object in a virtual space that the control object interacts with. In particular, it relates to detecting free-form gestures of a control object in a three-dimensional (3D) sensory space and generating for display a 3D solid control object model for the control object during the free-form gestures, including sub-components of the control object and in response to detecting a free-form gesture of the control object in the 3D sensory space in virtual contact with the virtual object, depicting, in the generated display, the virtual contact and resulting motions of the virtual object by the 3D solid control object model.

First claim

What is claimed is:

1. A method of manipulating virtual objects in a real time physics engine (RTPE) including a simulation of rigid bodies in a physical system that satisfies a human visual system's expectations for interactions with virtual objects in a virtual environment, using real motions of one or more hands in a three-dimensional (3D) sensory space, the method including:
determining a friction response to a soft contact collision between at least one of a set of virtual objects defined according to virtual object definitions and a portion of a colliding hand that collides in a logical frame defined by the RTPE by performing operations, including:
determining a first solution of interactions between virtual objects in the set of virtual objects simulated as rigid bodies and a representation of at least one hand, the first solution including the friction response to the soft contact collision between the at least one of the set of virtual objects and the portion of the colliding hand, the friction response being in a direction that is different than a direction of motion being undertaken by the portion of the colliding hand;
determining a second solution of interactions between the virtual objects simulated as the rigid bodies absent any effects of the hand; and
integrating the first solution with the second solution; wherein results of the second solution are prioritized over results of the first solution; and

determining a motion to apply to at least one virtual object as a rigid body based upon the integrating of the first solution with the second solution.

2. The method of claim 1 , further implementing the friction response with a direction opposite to a velocity of a hand portion colliding with a virtual object encountering a soft contact.

3. The method of claim 1 , further implementing the friction response with a magnitude proportional to a velocity of a hand portion colliding with a virtual object encountering a soft contact.

4. The method of claim 1 , further implementing the friction response with a magnitude set to a defined selected amount.

5. The method of claim 4 , wherein the defined selected amount is selected to be larger than other forces simulated by the RTPE.

6. The method of claim 1 , further including the first solution providing expected resultant velocities for virtual objects including at least one expected resultant velocity of at least one virtual object in soft contact with a portion of a hand colliding with the virtual object.

7. The method of claim 1 , wherein the first solution provides expected resultant velocities for virtual objects including at least one expected resultant velocity of at least one virtual object in soft contact with the portion of a hand colliding with the virtual object, further including:
the second solution discarding results of the first solution whenever attributing the expected resultant velocity to a virtual object causes the virtual object to lose physical integrity.

8. The method of claim 1 , further including capturing the set of captured images of one or more hands in the a three-dimensional (3D) sensory space and sensing a location of at least one hand using a video capturing sensor including at least one camera.

9. The method of claim 1 , further including performing the determining of the first solution in a first RTPE and the determining of the second solution in a second RTPE, the first RTPE being different from the second RTPE.

10. The method of claim 1 , further including permitting a portion of the colliding hand to partially penetrate a boundary defining a surface of a virtual object during a soft contact.

11. The method of claim 10 , further including: simulating in a brush contact phase a non-soft contact including a frictional force parallel to a surface of a virtual object and between at least one portion of a hand and a surface of the virtual object; and wherein the portion of the hand moves along and approximately parallel to the surface of the virtual object;
detecting a penetration by the portion of the hand into the virtual object exceeding a specified tolerance penetration for the portion of the hand;
responsive to the detecting a penetration exceeding the specified tolerance penetration, switching simulation for the portion of the hand, the virtual object and any other portions of the hand within a specified radius into soft contact collision simulation including the determining the first solution, the determining the second solution and the integrating the first solution with the second solution.

12. The method of claim 11 , further including starting a timer; and reverting to the brush contact phase when expiry of the timer occurs indicating a state in which no portion of the hand is touching the virtual object.

13. The method of claim 1 , the determining the first solution including receiving positions, velocities and geometry of virtual objects and portions of at least one hand and returning velocities of virtual objects responsive to the hand.

14. The method of claim 1 , further implementing presenting across a display of a head mounted device a display of the hand and the virtual object as a rigid body.

15. The method of claim 1 , wherein the representation comprises a capsule representation.

16. The method of claim 1 , wherein the friction response comprises a one-dimensional friction response.

17. The method of claim 1 , further implementing conducting first and second simulations in a subframe of a logical frame of the RTPE.

18. A non-transitory computer readable medium impressed with instructions for manipulating virtual objects in a real time physics engine (RTPE) including a simulation of rigid bodies in a physical system that satisfies a human visual system's expectations for interactions with virtual objects in a virtual environment, using real motions of one or more hands in a three-dimensional (3D) sensory space, which instructions, when executed by one or more processors, perform:
determining a friction response to a soft contact collision between at least one of a set of virtual objects defined according to virtual object definitions and a portion of a colliding hand that collides in a logical frame defined by the RTPE by performing operations, including:
determining a first solution of interactions between virtual objects in the set of virtual objects simulated as rigid bodies and a representation of at least one hand, the first solution including the friction response to the soft contact collision between the at least one of the set of virtual objects and the portion of the colliding hand, the friction response being in a direction that is different than a direction of motion being undertaken by the portion of the colliding hand;
determining a second solution of interactions between the virtual objects simulated as the rigid bodies absent any effects of the hand; and
integrating the first solution with the second solution; wherein results of the second solution are prioritized over results of the first solution; and

determining a motion to apply to at least one virtual object as a rigid body based upon the integrating of the first solution with the second solution.

19. A system including: an imaging sensor and a controller including one or more processors and a computer readable medium storing instructions for manipulating virtual objects in a real time physics engine (RTPE) including a simulation of rigid bodies in a physical system that satisfies a human visual system's expectations for interactions with virtual objects in a

Assignees

• Ultrahaptics Ip Two Ltd

Inventors

• Johnston John Adrian Arthur
• Selstad Johnathon Scott
• Marcolina Alex

Classifications

• G02B27/0176 Primary

  characterised by mechanical features · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T2210/21

  Collision detection, intersection · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title