Mixed reality holographic object development

Patent metadata

Field	Value
Publication number	US-9429912-B2
Application number	US-201213588962-A
Country	US
Kind code	B2
Filing date	Aug 17, 2012
Priority date	Aug 17, 2012
Publication date	Aug 30, 2016
Grant date	Aug 30, 2016

Abstract

Systems and related methods for presenting a holographic object that self-adapts to a mixed reality environment are provided. In one example, a holographic object presentation program captures physical environment data from a destination physical environment and creates a model of the environment including physical objects having associated properties. The program identifies a holographic object for display on a display of a display device, the holographic object including one or more rules linking a detected environmental condition and/or properties of the physical objects with a display mode of the holographic object. The program applies the one or more rules to select the display mode for the holographic object based on the detected environmental condition and/or the properties of the physical objects.

First claim

The invention claimed is:

1. A self-adapting holographic object presentation system for presenting a holographic object that self-adapts to a mixed reality environment including a destination physical environment and a virtual environment, the self-adapting holographic object presentation system comprising:
a display device including an associated processor and memory;
a holographic object presentation program executed by the processor using portions of the memory, the holographic object presentation program configured to:
capture physical environment data from the destination physical environment using one or more sensors;
create a model of the destination physical environment based on the captured physical environment data, the model including identified physical objects in the destination physical environment having associated physical object properties;
identify a holographic object for display on the display device, wherein the holographic object includes one or more rules linking a detected environmental condition and/or the physical object properties of the identified physical objects with a display mode of the holographic object, wherein the display mode comprises a first user interaction mode that is based on a first display resolution of the holographic object that is above a threshold resolution and a second user interaction mode that is based on a second display resolution of the holographic object that is below the threshold resolution;
apply the one or more rules to select the display mode of the holographic object based on the detected environmental condition and/or the physical object properties of the identified physical objects in the destination physical environment; and
display the holographic object on the display device according to the display mode.

2. The self-adapting holographic object presentation system of claim 1 , wherein the one or more rules are contained within a data schema for the holographic object.

3. The self-adapting holographic object presentation system of claim 1 , wherein the sensors are selected from the group consisting of eye-tracking sensors, optical sensors, and positions sensors, and wherein creating the model of the destination physical environment includes a surface reconstruction phase in which surfaces are identified and reconstructed, and an object identification phase in which the identified surfaces are grouped into the identified physical objects, with the physical object properties of the identified physical objects being either prestored or identified through the physical environment data.

4. The self-adapting holographic object presentation system of claim 1 , wherein the display mode of the holographic object includes a scale factor that indicates a scale at which the holographic object will be rendered relative to the destination physical environment.

5. The self-adapting holographic object presentation system of claim 1 , wherein the display mode of the holographic object includes an auto-fit factor that indicates that the holographic object will be automatically sized to fit into the destination physical environment.

6. The self-adapting holographic object presentation system of claim 1 , wherein the display mode includes a user interaction factor that indicates the user interaction mode for the holographic object.

7. The self-adapting holographic object presentation system of claim 1 , wherein the holographic object presentation system is configured to download the holographic object from a library of holographic objects via a computer network.

8. A method for presenting a holographic object that self-adapts to a mixed reality environment including a destination physical environment and a virtual environment, comprising:
capturing physical environment data from the destination physical environment using one or more sensors of a display device;
creating a model of the destination physical environment based on the captured physical environment data, the model including identified physical objects in the destination physical environment having associated physical object properties;
identifying a holographic object for display on the display device, wherein the holographic object includes one or more rules linking a detected environmental condition and/or physical object properties of the identified physical objects with a display mode of the holographic object, wherein the display mode comprises a first user interaction mode that is based on a first display resolution of the holographic object that is above a threshold resolution and a second user interaction mode that is based on a second display resolution of the holographic object that is below the threshold resolution;
applying the one or more rules to select the display mode of the holographic object based on the detected environmental condition and/or the physical object properties of the identified physical objects in the destination physical environment; and
displaying the holographic object on a display of the display device according to the display mode.

9. The method of claim 8 , wherein the one or more rules are contained within a rules data schema for the holographic object.

10. The method of claim 8 , wherein the sensors are selected from the group consisting of eye-tracking sensors, optical sensors, and positions sensors, and wherein creating the model of the destination physical environment further includes:
identifying and reconstructing surfaces in a surface reconstruction phase, and
grouping the identified surfaces into the identified physical objects in an object identification phase, with the physical object properties of the identified physical objects being either prestored or identified through the physical environment data.

11. The method of claim 8 , wherein the rules are developer-specified or user-specified.

12. The method of claim 8 , wherein the display mode of the holographic object includes a scale factor that indicates a scale at which the holographic object will be rendered relative to the destination physical environment.

13. The method of claim 8 , wherein the display mode of the holographic object includes an auto-fit factor that indicates that the holographic object will be automatically sized to fit into the destination physical environment.

14. The method of claim 8 , wherein the display mode includes a user interaction factor that indicates the user interaction mode for the holographic object.

15. A method for developing a holographic object that self-adapts to a mixed reality environment including a virtual environment and a destination physical environment with physical objects having associated physical object properties, comprising:
determining a physical environment data schema that includes the physical object properties and associated value ranges, where each of a plurality of possible destination physical environments is representable by an instance of the physical environment data schema populated with specific physical data values for each of the physical object properties;
determining a virtual environment data schema that includes virtual environment characteristics and associated value ranges, where each of a plurality of possible virtual environments is representable by an instance of the virtual environment data schema populated with specific virtual data values for each of the virtual environment characteristics;
displaying a holographic object creation interface on a display, the holographic object creation interface being configured to display at least one selector for receiving user input corresponding to a display mode of the holographic obj

Assignees

• Fleck Rod G
• Kamuda Nicholas
• Latta Stephen
• Kinnebrew Peter Tobias
• Microsoft Technology Licensing Llc

Inventors

• Fleck Rod G
• Kamuda Nicholas
• Latta Stephen
• Kinnebrew Peter Tobias

Classifications

• G03H2001/0061

  in haptic applications when the observer interacts with the holobject · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G03H2001/2284

  Superimposing the holobject with other visual information · CPC title

• G03H1/2249 Primary

  Holobject properties · CPC title

• G02B27/0103

  comprising holographic elements · CPC title