Augmented reality simulation continuum

Patent metadata

Field	Value
Publication number	US-10282882-B2
Application number	US-201715658580-A
Country	US
Kind code	B2
Filing date	Jul 25, 2017
Priority date	Jun 29, 2012
Publication date	May 7, 2019
Grant date	May 7, 2019

Abstract

Techniques for simulating interactions using an augmented reality device. A visual scene is captured using a camera device. The visual scene depicts of a first physical object within a physical environment. A dynamic interaction is simulated between the first physical object and one or more virtual objects and velocity of a first virtual object resulting from the simulated dynamic interaction is calculated. A predefined behavior for a virtual character to perform is selected in response to the dynamic interaction. The virtual character is distinct from the first virtual object, and the selection is based on the calculated velocity. A sequence of frames is rendered. The sequence of frames depicts the dynamic interaction between the first physical object and the virtual objects, and further depicts a representation of the virtual character performing the determined behavior. The sequence of frames is output for display using one or more display devices.

First claim

What is claimed is:

1. A computer-implemented method for simulating interactions using an augmented reality device, the computer-implemented comprising:
capturing, using one or more camera devices, a visual scene that includes a depiction of a first physical object within a physical environment;
rendering a first sequence of frames that depicts a virtual character interacting with at least one of the first physical object and one or more virtual objects, via a behavior of a first type;
simulating, by operation of one or more computer processors of the augmented reality device, a dynamic interaction between the first physical object and the one or more virtual objects;
calculating a velocity of a first one of the one or more virtual objects, resulting from the simulated dynamic interaction;
determining, based on the velocity resulting from the simulated dynamic interaction, a behavior of a second type different than the first type, for the virtual character to perform;
rendering a second sequence of frames that depicts the dynamic interaction between the first physical object and the one or more virtual objects and further depicts the virtual character performing the behavior of the second type; and
outputting the first and second sequences of frames for display using one or more display devices.

2. The computer-implemented method of claim 1 , further comprising:
identifying predefined geometry information for the first physical object, wherein the predefined geometry information includes at least a measure of size for the first physical object; and
estimating a depth measure of the physical object within the visual scene, based on the identified predefined geometry information,
wherein simulating the dynamic interaction is further based on the estimated depth measure of the first physical object.

3. The computer-implemented method of claim 1 , further comprising:
monitoring a motion of the first physical object within the visual scene;
estimating a velocity of the first physical object based on the monitored motion; and
determining a velocity of the one or more virtual objects to be displayed within the visual scene,
wherein the dynamic interaction is further based on the determined velocity of the first physical object and the determined velocity of the one or more virtual objects.

4. The computer-implemented method of claim 1 , wherein simulating a dynamic interaction between the first physical object and the one or more virtual objects is further based on a set of dynamics rules that define dynamics interactions for visual scenes displayed on the augmented reality device.

5. The computer-implemented method of claim 1 , further comprising:
applying a force to the first physical object outside of the captured visual scene, based on the simulated dynamic interaction between the physical object and the one or more virtual objects within the visual scene, comprising transmitting an instruction to a control device, to cause the control device to activate an actuator associated with a first physical device in order to move the first physical object at a specified speed and trajectory, wherein the speed and trajectory are dynamically determined based on the dynamic interaction between the physical object and the one or more virtual objects within the visual scene.

6. The computer-implemented method of claim 1 , wherein the virtual character comprises an animated character, wherein the behaviors of the first and second types interact with only one of the first physical object and the one or more virtual objects, wherein the dynamic interaction comprises a collision that does not include the animated character, wherein the character performs one type of behavior at a time, wherein in performing the behavior of the second type, the character ceases performing the behavior of the first type, wherein the animated character comprises an anthropomorphic character, wherein the virtual character is distinct from the one or more virtual objects, wherein the computer-implemented method further comprises:
identifying predefined geometry information for the first physical object, wherein the predefined geometry information includes at least a measure of size for the first physical object; and
estimating a depth measure of the physical object within the visual scene, based on the identified predefined geometry information,
wherein simulating the dynamic interaction is further based on the estimated depth measure of the first physical object.

7. The computer-implemented method of claim 6 , further comprising:
monitoring a motion of the first physical object within the visual scene;
estimating a velocity of the first physical object based on the monitored motion; and
determining a velocity of the one or more virtual objects to be displayed within the visual scene,
wherein the dynamics simulation is further based on the determined velocity of the first physical object and the determined velocity of the one or more virtual objects;
wherein simulating a dynamic interaction between the first physical object and the one or more virtual objects is further based on a set of dynamics rules that define dynamics interactions for visual scenes displayed on the augmented reality device.

8. The computer-implemented method of claim 7 , further comprising:
applying a force to the first physical object outside of the captured visual scene, based on the simulated dynamic interaction between the physical object and the one or more virtual objects within the visual scene, comprising transmitting an instruction to a control device, to cause the control device to activate an actuator associated with a first physical device in order to move the first physical object at a specified speed and trajectory, wherein the speed and trajectory are dynamically determined based on the dynamic interaction between the physical object and the one or more virtual objects within the visual scene.

9. The computer-implemented method of claim 8 , further comprising:
determining a surface within the physical environment on which the first physical object is positioned;
retrieving a predefined profile corresponding to the surface, wherein the predefined profile described a plurality of attributes of the surface;
simulating a dynamic interaction between the first physical object and one or more virtual objects, based at least in part on the plurality of attributes of the surface described in the predefined profile;
transmitting a message to a controller associated with the first physical object outside of the captured visual scene, wherein the controller is configured to apply the force to the first physical object in response to receiving the message;
wherein the first physical object is substantially spherical in shape, wherein the actuator comprises a gyroscope disposed within the first physical object that, when activated, dynamically alters a trajectory of the first physical object such that the first physical object moves at the calculated velocity.

10. The computer-implemented method of claim 9 , wherein the first physical object is spherical in shape, wherein the first sequence of frames comprises an initial sequence of frames that depicts the virtual character performing the behavior of the first type, the behavior of the first type comprising balancing atop the first physical object, wherein the behavior of the second type comprises the virtual character falling from atop the first physical object, wherein the computer-implemented method further comprises:
monitoring movement of the first physical object to detect a collision; and
determining that the first spherical physical object is moving at the calculated velocity as a result of t

Assignees

• Disney Entpr Inc

Inventors

• Sumner Robert
• Mitchell Kenneth J
• Hostettler Rafael C
• Coros Stelian

Classifications

• G06T2210/21

  Collision detection, intersection · CPC title

• G06T13/20 Primary

  Three-dimensional [3D] animation · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• A63F13/25

  Output arrangements for video game devices · CPC title

• A63F2300/8082

  Virtual reality · CPC title