Object-centric mixed reality space

Patent metadata

Field	Value
Publication number	US-9767720-B2
Application number	US-201213532636-A
Country	US
Kind code	B2
Filing date	Jun 25, 2012
Priority date	Jun 25, 2012
Publication date	Sep 19, 2017
Grant date	Sep 19, 2017

Abstract

A see-through, near-eye, mixed reality display apparatus providing a mixed reality environment wherein one or more virtual objects and one or more real objects exist within the view of the device. Each of the real and virtual have a commonly defined set of attributes understood by the mixed reality system allowing the system to manage relationships and interaction between virtual objects and other virtual objects, and virtual and real objects.

First claim

What is claimed is:

1. A method presenting a mixed reality environment allowing viewing of real world objects integrated with virtual objects by a user, comprising:
determining one or more real objects viewable by the user within a user environment;
determining one or more three-dimensional virtual objects adapted to be rendered to the user and viewable in conjunction with the one or more real objects within the user environment;
rendering the one or more virtual objects within a user field of view within the environment;
mapping a real object of the one or more real objects and a three-dimensional virtual object of the one or more three-dimensional virtual objects to respective object instances, the three-dimensional virtual object and the real object in the object instances defined using the same object definition including the same object definition of a physical attribute of the one or more real objects and the one or more three-dimensional virtual objects; and
managing interaction between the three-dimensional virtual object and the real object based on the physical attribute defined in the respective object instances upon interaction of the three-dimensional virtual object and the real object.

2. The method of claim 1 wherein said rendering of the one or more virtual objects is based on at least one relational attribute in the same object definition defining a behavior of the one or more virtual objects relative to the one or more real objects.

3. The method of claim 1 wherein the step of managing interaction comprises:
tracking the real object and the virtual object within the user environment;
determining a virtual-real object interaction when the virtual object interacts with the real object; and
rendering the virtual-real object interaction based on the set of attributes for a virtual object instance and the attributes of a real object instance.

4. The method of claim 1 further including:
tracking each of the one or more real objects and each of the one or more virtual objects within the user environment;
determining a virtual-virtual object interaction when a virtual object interacts with another virtual object;
rendering the virtual-virtual object interaction based on a user filter;
determining a virtual-real object interaction when a virtual object interacts with a real object; and
rendering the virtual-real object interaction based on the set of attributes for a virtual object instance and the attributes of a real object instance.

5. The method of claim 1 wherein the set of attributes includes one or more functions for the virtual object.

6. The method of claim 1 further including generating the object instances and sharing object instances with other users via a communication link, and receiving shared object instances from other users, and including rendering and managing the interaction between the object instances shared by other users and generated object instances.

7. A see through head mounted display apparatus, comprising:
a see-through, near-eye, augmented reality display adapted to render three-dimensional virtual objects to the user in the display which are viewable in conjunction with real objects;
one or more processing devices in wireless communication with apparatus, the one or more processing devices automatically determine an environment, one or more real objects in the environment and one or more three-dimensional virtual objects in the environment, the one or more processing devices assign an object instance to each of the real and three-dimensional virtual objects in the environment, each three-dimensional virtual object and each real object in each object instance defined having an object definition representing a physical trait of the one or more three-dimensional virtual objects and the one or more real objects and provided in a data structure containing a common set of attributes for the real and three-dimensional virtual objects such that each three-dimensional virtual object and each real object share the same attributes, the one or more processing devices determine input data from real world objects and three-dimensional virtual objects in a field of view and integrate interaction between real and three-dimensional virtual objects based on the object instances; wherein said interaction of the one or more three-dimensional virtual objects with one or more real objects is based on at least one relational attribute in the common set of attributes defining the behavior of the three-dimensional virtual object relative to the one or more real objects.

8. The apparatus of claim 7 wherein the common set of attributes comprises a data structure including includes at least one attribute of:
object type, spatial coordinates, object registration, reality rating, dynamic scaling, ownership, user permissions, content rating, physical properties, learned attributes, related objects and functions.

9. The apparatus of claim 8 wherein the physical properties include at least physics attributes defining object movement and actions and an interaction rule set defining object interaction with other objects.

10. The apparatus of claim 8 wherein the object definition includes an identifier.

11. The apparatus of claim 10 wherein each instance of an object definition is specifically identified.

12. The apparatus of claim 8 wherein the apparatus includes a memory and a data structure, the data structure including one or more object definitions modified by a user and owned by a user.

13. The apparatus of claim 12 wherein the one or more processors
track each of the real objects and each of the virtual objects within a user environment;
determine a virtual-virtual object interaction when a virtual object interacts with another virtual object;
render the virtual-virtual object interaction based on a user filter;
determine a virtual-real object interaction when a virtual object interacts with another virtual object; and
render the virtual-real object interaction based on the set of attributes for a virtual object instance and the attributes of a real object instance.

14. A method for managing interaction between virtual holographic objects and real world objects in a mixed reality environment generated by a see through head mounted display system, comprising:
determining an environment and orientation of the system, the system includes one or more sensors and a see-through display;
determining three-dimensional locations in three-dimensional space of real and virtual objects within an environment of a wearer of the see-through display in the environment;
creating an object instance for virtual objects and real objects within the environment, the virtual objects and the real objects in the object instances defined based on a common object definition comprising a set of attributes such that the virtual objects and the real objects share the same sub-set of core attributes, wherein the set of attributes of the common object definition relates to a common physical trait of three-dimensional virtual object and each real object;
determining whether an interaction between at least two objects occurs, the interaction being one of an interaction between a virtual object and another virtual object, or an interaction between a virtual object and a real world object, and the interaction comprising the at least two objects sharing at least one point in three-dimensional space;
rendering virtual objects to the display which are viewable in conjunction with the real objects where the interaction between the at least two objects in the display based on attributes defined in the object instance of any intera

Assignees

• Kinnebrew Peter Tobias
• Kamuda Nicholas Ferianc
• Microsoft Technology Licensing Llc

Inventors

• Kinnebrew Peter Tobias
• Kamuda Nicholas Ferianc

Classifications

• G09G2340/12

  Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels · CPC title

• G09G2340/125

  wherein one of the images is motion video · CPC title

• G09G3/003 Primary

  to produce spatial visual effects · CPC title