Augmented reality device with predefined object data

Patent metadata

Field	Value
Publication number	US-10217289-B2
Application number	US-201715839542-A
Country	US
Kind code	B2
Filing date	Dec 12, 2017
Priority date	Mar 15, 2013
Publication date	Feb 26, 2019
Grant date	Feb 26, 2019

Abstract

Embodiments capture one or more images of a visual scene that includes a first physical object. A first region of the first physical object to apply one or more augmentations to is determined. Embodiments determine which configuration the first physical object is currently in. The first physical object is configured to be physically manipulatable into each of a plurality of configurations. A sequence of frames is rendered for display in which the first region of the first physical object is animated in a predefined manner depicting a virtual light source within the first physical object, based on the determined configuration of the first physical object, by applying the one or more augmentations to a first virtual object generated based on predefined geometric information corresponding to a determined object type of the first physical object. The rendered sequence of frames is output for display using one or more display devices.

First claim

What is claimed is:

1. A computer-implemented method to augment virtual objects corresponding to physical objects, based on type and configuration of the physical objects, the computer-implemented method comprising:
capturing, using one or more camera devices, one or more images of a visual scene that includes a first physical object;
classifying the first physical object as a first predetermined object type, based on at least one of: (i) one or more object identifiers associated with the first physical object and (ii) a plurality of edges of the first physical object within the visual scene;
determining which one of a plurality of distinct physical configurations the first physical object is currently in, wherein the first physical object is configured to be physically manipulatable into each of the plurality of distinct physical configurations;
generating a first virtual object based on predefined geometric information corresponding to the first predetermined object type of the first physical object;
determining one or more augmentations to apply to a first region of the first virtual object, by operation of one or more computer processors and based on the predefined geometric information corresponding to the first predetermined object type of the first object;
rendering a sequence of frames for display in which the first region of the first physical object is animated in a predefined manner depicting a virtual light source within the first physical object, including applying, based on the determined physical configuration of the first physical object, the one or more augmentations to the first virtual object; and
outputting the rendered sequence of frames for display using one or more display devices.

2. The computer-implemented method of claim 1 , further comprising:
analyzing the visual scene in order to determine the plurality of edges of the first physical object within the visual scene.

3. The computer-implemented method of claim 1 , further comprising:
measuring one or more environmental illumination characteristics of an environment in which an augmented reality device is located, wherein the environmental illumination characteristics include at least one of a position of a light source within an environment in which the augmented reality device is located, an angle of the light source, an indication of whether the light source is omnidirectional, a color of the light source, an intensity of the light source and a reflectivity value of the first physical object; and
adjusting the appearance of the augmented first virtual object based on the measured one or more environmental illumination characteristics.

4. The computer-implemented method of claim 1 , wherein the visual scene is augmented to include one or more virtual objects, and further comprising:
determining one or more illumination effects associated with a first one of the one or more virtual objects,
wherein the visual scene is augmented further based on the determined one or more illumination effects.

5. The computer-implemented method of claim 1 , further comprising:
receiving a signal associated with the first physical object; and
processing the received signal in order to determine the one or more object identifiers that are encoded within the signal.

6. The computer-implemented method of claim 1 ,
wherein classifying the first physical object as the first predetermined object type comprises:
detecting an embedded marker within the first physical object; and
determine the first physical object corresponds to the first predetermined object type, based on the detected embedded marker.

7. The computer-implemented method of claim 1 , wherein each distinct physical configuration has a distinct physical shape, wherein the computer-implemented method comprises:
analyzing the visual scene in order to determine the plurality of edges of the first physical object within the visual scene;
measuring one or more environmental illumination characteristics of an environment in which an augmented reality device is located, wherein the environmental illumination characteristics include at least one of a position of a light source within an environment in which the augmented reality device is located, an angle of the light source, an indication of whether the light source is omnidirectional, a color of the light source, an intensity of the light source, and a reflectivity value of the first physical object; and
adjusting the appearance of the augmented first virtual object based on the measured one or more environmental illumination characteristics.

8. The computer-implemented method of claim 7 , wherein the visual scene is augmented to include one or more virtual objects, and further comprising:
determining one or more illumination effects associated with a first one of the one or more virtual objects,
wherein the visual scene is augmented further based on the determined one or more illumination effects.

9. The computer-implemented method of claim 8 , further comprising:
receiving a signal associated with the first physical object; and
processing the received signal in order to determine the one or more object identifiers that are encoded within the signal.

10. The computer-implemented method of claim 9 , wherein classifying the first physical object as the first predetermined object type comprises:
detecting an embedded marker within the first physical object; and
determine the first physical object corresponds to the first predetermined object type, based on the detected embedded marker, wherein the embedded marker is embedded in the first physical object using an infrared-absorbing material, wherein the embedded marker is detected using an infrared camera coupled to the augmented reality device.

11. The computer-implemented method of claim 10 , further comprising:
inserting an animated virtual character into the visual scene, wherein the animated virtual character is depicted in the rendered sequence of frames as interacting with the first physical object in the rendered sequence of frames, based on the predefined geometric information;
wherein the interaction between the animated virtual character and the first physical object is based on a set of dynamics rules that define dynamics interactions for visual scenes displayed on the augmented reality device.

12. The computer-implemented method of claim 11 , wherein the environmental illumination characteristics include the position of the light source within the environment in which the augmented reality device is located, the angle of the light source, the indication of whether the light source is omnidirectional, the color of the light source, the intensity of the light source, and the reflectivity value of the first physical object.

13. A computer-readable memory containing a program that, when executed, performs an operation to augment virtual objects corresponding to physical objects, based on type and configuration of the physical objects, the operation comprising:
capturing, using one or more camera devices, one or more images of a visual scene that includes a first physical object;
classifying the first physical object as a first predetermined object type, based on one or more object identifiers associated with the first physical object;
determining which one of a plurality of distinct physical configurations the first physical object is currently in, wherein the first physical object is configured to be physically manipulatable into each of the plurality of distinct physical configurations;
generating a first virtual object based on predefined geometric information corresponding to the first predeterm

Assignees

• Disney Entpr Inc

Inventors

• Mitchell Kenneth John
• Sumner Robert Walker

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title