Dynamic augmentation of a physical scene

Patent metadata

Field	Value
Publication number	US-9761059-B2
Application number	US-201414269782-A
Country	US
Kind code	B2
Filing date	May 5, 2014
Priority date	Jan 3, 2014
Publication date	Sep 12, 2017
Grant date	Sep 12, 2017

Abstract

Computer-readable storage media, computing device and methods associated with dynamic modification of a rendering of a physical scene. In embodiments, one or more computer-readable storage media may have instructions stored thereon which, when executed by a computing device, may provide the computing device with a dynamic augmentation module. The dynamic augmentation module may, in some embodiments, cause the computing device to receive a manipulation of a physical scene. In response to receipt of the manipulation, the dynamic augmentation module may cause the computing device to dynamically modify a rendering of the physical scene. In some embodiments, this may be accomplished through real-time application of one or more virtual articles to the rendering of the physical scene or alteration of one or more virtual articles added to the rendering of the physical scene. Other embodiments may be described and/or claimed.

First claim

What is claimed is:

1. A computing device comprising:
a display to render a physical scene;
one or more computing processors coupled with the display;
physical memory coupled with the one or more processors
a plane calculation module, to be loaded into the physical memory and executed by the one or more processors, to generate a rendering of the physical scene on the display, wherein the plane calculation module is to:
receive one or more indications of physical objects associated with the physical scene; and
based upon the received one or more indications, generate a rendering of the physical scene; and

a scene augmentation module, to be loaded into the physical memory and executed by the one or more processors, to aid in completion of a performance of a task, wherein the scene augmentation module is to:
detect a sub task of a manipulation of the physical scene by adding, moving, or removing a physical object in the scene, wherein at least one predefined physical object has an associated context relating to the sub task of the task;
dynamically modify the rendering of the physical scene, through real-time application of one or more virtual articles to the rendering of the physical scene or alteration of one or more virtual articles added to the rendering of the physical scene, based on the manipulation, wherein the modification of the rendering is to aid a user in completing the performance of the task; and
upon completion of the performance of the task, dynamically modify the rendering of the physical scene through real-time application of a virtual article to interact with a rendering of a physical object related to the completion of the performance of the task, wherein the interaction indicates the completion of the task.

2. The computing device of claim 1 , wherein the rendering of the physical scene is a three dimensional, 3-D, rendering; and
wherein to dynamically modify the rendering of the physical scene further comprises to:
identify a predefined physical object within the 3-D rendering of the physical scene; and
apply one or more virtual articles to the 3-D rendering of the physical scene or alter one or more virtual articles added to the 3-D rendering of the physical scene, based on the identified predefined physical object.

3. The computing device of claim 2 , wherein color information is contained within a color image of the physical scene and depth information is contained within a depth image of the physical scene and wherein the computing device further comprises an image processing module coupled with the scene augmentation module to align the color information and the depth information for further processing.

4. The computing device of claim 3 , wherein to identify the predefined physical object, the scene augmentation module is further to:
analyze a plurality of surfaces within the rendering of the physical scene to determine surfaces of the rendering of the physical scene that have connected continuity indicative of the surfaces being a part of a single physical object;
extract features of the single physical object for comparison against known features of a plurality of predefined physical objects; and
compare the extracted features with features of the plurality of predefined physical objects to identify the single physical object as one of the plurality of predefined physical objects.

5. The computing device of claim 2 , wherein to apply or alter one or more virtual articles is further based on a context associated with the predefined physical object.

6. The computing device of claim 5 , wherein the context associated with the predefined physical object corresponds with one or more of:
a previous location of the predefined physical object;
another predefined physical object within the physical scene; or
one or more virtual articles previously applied to the physical object or the physical scene.

7. The computing device of claim 6 , wherein the context associated with the predefined physical object corresponds with a previous location of the predefined physical object indicating movement, with respect to the physical scene, of the predefined physical object, and wherein to dynamically modify the 3-D rendering of the physical scene is to modify one or more virtual articles based on the indicated movement.

8. The computing device of claim 7 , wherein the context associated with the predefined physical object corresponds with another predefined physical object within the physical scene, and wherein to dynamically modify the 3-D rendering of the physical scene is to modify one or more virtual articles based on the another predefined physical object.

9. The computing device of claim 1 , wherein completing a task includes solving a puzzle or completing a computer-based learning exercise.

10. One or more non-transitory computer-readable storage media having instructions stored thereon which, when executed by a computing device, provide the computing device with a dynamic augmentation module to receive a manipulation of a physical scene, and in response to the receipt, dynamically modify a rendering of the physical scene, through real-time application of one or more virtual articles to the rendering of the physical scene or alteration of one or more virtual articles added to the rendering of the physical scene;
wherein the manipulation of the physical scene comprises addition of a predefined physical object to the physical scene,
wherein to dynamically modify the rendering of the physical scene comprises application of one or more virtual articles to the rendering of the physical scene based upon the predefined physical object, wherein at least one virtual article has an associated context relating to a sub task of a performance of a task, wherein the modification of the rendering is to aid a user in completing the performance of the task, and
wherein upon completion of the performance of the task, application of the one or more virtual articles to a real-time interaction with a rendering of a physical object is related to the completion of the performance of the task, wherein the interaction indicates the completion of the task.

11. The one or more non-transitory computer-readable storage media of claim 10 , wherein to dynamically modify the rendering of the physical scene, the dynamic augmentation module is further to:
receive color information and depth information associated with the physical scene;
generate a three dimensional, 3-D, rendering of the physical scene from the color and depth information;
identify a predefined physical object within the 3-D rendering of the physical scene; and
apply one or more virtual articles to the 3-D rendering of the physical scene or alter one or more virtual articles added to the 3-D rendering of the physical scene, based on the identified predefined physical object.

12. The one or more non-transitory computer-readable storage media of claim 11 , wherein the color information is contained within a color image of the physical scene and the depth information is contained within a depth image of the physical scene.

13. The one or more non-transitory computer-readable storage media of claim 12 , wherein to identify the predefined physical object, the dynamic augmentation module is further to:
analyze a plurality of surfaces within the physical scene utilizing the color image and the depth image to determine components of the physical scene that have connected continuity indicative of the components being a part of a single physical object;
extract features of the single physical object for comparison against known features of a plurality of predefined physical objects; and

Assignees

• Intel Corp

Inventors

• Nistel Kobi
• Hurwitz Barak
• Kamhi Gila
• Reif Dror
• Cooperman Vladimir

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T15/04

  Texture mapping · CPC title

• G06T2219/2021

  Shape modification · CPC title

• G06T2215/06

  Curved planar reformation of 3D line structures · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title