Method and system for compositing an augmented reality scene

Patent metadata

Field	Value
Publication number	US-9824495-B2
Application number	US-200913063724-A
Country	US
Kind code	B2
Filing date	Sep 13, 2009
Priority date	Sep 11, 2008
Publication date	Nov 21, 2017
Grant date	Nov 21, 2017

Abstract

Disclosed are systems and methods for compositing an augmented reality scene, the methods including the steps of extracting, by an extraction component into a memory of a data-processing machine, at least one object from a real-world image detected by a sensing device; geometrically reconstructing at least one virtual model from at least one object; and compositing AR content from at least one virtual model in order to augment the AR content on the real-world image, thereby creating AR scene. Preferably, the method further includes; extracting at least one annotation from the real-world image into the memory of the data-processing machine for modifying at least one virtual model according to at least one annotation. Preferably, the method further includes: interacting with AR scene by modifying AR content based on modification of at least one object and/or at least one annotation in the real-world image.

First claim

What is claimed is:

1. A system for compositing an augmented reality (AR) scene, the system comprising:
(a) an extraction component, operationally connected to a sensing device for capturing a real-world image, to:
(i) extract an object from said real-world image into a memory of a data-processing machine, wherein the object defines how to geometrically construct a virtual model of the object according to a predefined rule set;
(ii) geometrically construct the virtual model from said object, wherein geometrically constructing the virtual model comprises generating a geometry and texture of the virtual model based on an interpretation of the extracted object according to the predefined rule set, and wherein geometrically constructing the virtual model comprises modifying a property of the virtual model, such that an appearance or behavior of the virtual model upon rendering is different from a corresponding appearance or behavior of the object in the real-world image; and
(iii) composite AR content from said constructed virtual model in order to augment said AR content on said real-world image, thereby creating the AR scene.

2. The system of claim 1 , wherein said extraction component further:
(iv) extracts at least one annotation from said real-world image into said memory of said data-processing machine for modifying the appearance or behavior of said virtual model in said real-world image according to at least one annotation.

3. The system of claim 1 , wherein geometrically constructing the virtual model is performed independently of a predefined content or a pre-existing virtual model.

4. The system of claim 1 , wherein said real-world image includes at least one image type selected from the group consisting of: a printed image, a hand-sketched image, a hand-drawn image, a sprayed image, an image displayed on a screen, an image projected on a surface, a weaved image, a carved-surface image, and an engraved-surface image.

5. The system of claim 1 , wherein said real-world image includes at least two correlated annotations for modifying an effect of other annotations.

6. The system of claim 2 , wherein said real-world image includes a legend comprising said annotation.

7. The system of claim 1 , wherein said object includes the entirety of said real-world image.

8. The system of claim 1 , wherein said extraction component includes at least one functional component selected from the group consisting of: an object-extractor component, a geometry-analysis component, a modeling-cues-analysis component, an animation-analysis component, a space-warp-analysis component, a physical-property-analysis component, a user-interaction-analysis component, an inter-object-connection-analysis component, a mesh-generation component, a texture-generation component, and an object-factory component.

9. The system of claim 1 , wherein said extraction component further (iv) interacts with the AR scene by modifying said AR content based on modification of said object and/or at least one annotation in said real-world image.

10. A method for compositing an augmented reality (AR) scene, the method comprising the steps of:
(a) extracting, by an extraction component into a memory of a data-processing machine, an object from a real-world image detected by a sensing device, wherein the object defines how to geometrically construct a virtual model of the object according to a predefined rule set;
(b) geometrically constructing the virtual model from said object, wherein geometrically constructing the virtual model comprises generating the geometry and texture of the virtual model based on an interpretation of the extracted object according to the predefined rule set, and wherein geometrically constructing the virtual model comprises modifying a property of the virtual model, such that an appearance or behavior of the virtual model upon rendering is different from a corresponding appearance or behavior of the object in the real-world image; and
(c) compositing AR content from said reconstructed virtual model in order to augment said AR content on said real-world image, thereby creating the AR scene.

11. The method of claim 10 , the method further comprising the step of:
(d) extracting at least one annotation from said real-world image into said memory of said data-processing machine for modifying the appearance or behavior of said virtual model in said real-world image according to at least one annotation.

12. The method of claim 10 , wherein said step of geometrically constructing the virtual model is performed independently of a predefined content or a pre-existing virtual model.

13. The method of claim 11 , wherein said real-world image includes a legend comprising said annotation.

14. The method of claim 10 , the method further comprising the step of:
(d) interacting with the AR scene by modifying said AR content based on modification of said object and/or at least one annotation in said real-world image.

15. A non-transitory computer-readable storage medium having computer-readable code embodied on the computer-readable storage medium, the computer-readable code comprising:
(a) program code for extracting, by an extraction component into a memory of a data-processing machine, an object from a real-world image detected by a sensing device, wherein the object defines how to geometrically construct a virtual model of the object according to a predefined rule set;
(b) program code for-geometrically constructing the virtual model from said object, wherein geometrically constructing the virtual model comprises generating the geometry and texture of the virtual model based on an interpretation of the extracted object according to the predefined rule set, and wherein geometrically constructing the virtual model comprises modifying a property of the virtual model, such that an appearance or behavior of the virtual model upon rendering is different from a corresponding appearance or behavior of the object in the real-world image; and
(c) program code for compositing AR content from said constructed virtual model in order to augment said AR content on said real-world image, thereby creating the AR scene.

16. A system for compositing an augmented reality (AR) scene, the system comprising:
(a) an extraction component, operationally connected to a sensing device for capturing a real-world image, to:
(i) extract at least one object from said real-world image into a memory of a data-processing machine, wherein the at least one object defines how to geometrically construct at least one virtual model and information relating to a behavior associated with the at least one virtual model according to a predefined rule set;
(ii) geometrically construct the at least one virtual model from said at least one object and from the information relating to a behavior associated with the at least one virtual model, wherein geometrically constructing the virtual model comprises generating the geometry and texture of the virtual model based on an interpretation of the extracted object according to the one or more predefined rules; and
(iii) composite AR content from said at least one constructed virtual model in order to augment said AR content on said real-world image, thereby creating the AR scene.

17. A method for compositing an augmented reality (AR) scene, the method comprising the steps of:
(a) extracting, by an extraction component into a memory of a data-processing machine, at least one object from a real-world image detected by a sensing device, wherein the at least one object defines, according to a predefined rules

Assignees

• Hagbi Netanel
• Bergig Oriel Y
• Elsana Jihad A
• Apple Inc

Inventors

• Hagbi Netanel
• Bergig Oriel Y
• Elsana Jihad A

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title