Mixed reality experience sharing

Patent metadata

Field	Value
Publication number	US-9342929-B2
Application number	US-201313747328-A
Country	US
Kind code	B2
Filing date	Jan 22, 2013
Priority date	Jan 22, 2013
Publication date	May 17, 2016
Grant date	May 17, 2016

Abstract

Embodiments that relate to presenting a textured shared world model of a physical environment are disclosed. One embodiment includes receiving geo-located crowd-sourced structural data items of the physical environment. The structural data items are stitched together to generate a 3D spatial shared world model. Geo-located crowd-sourced texture data items are also received and include time-stamped images or video. User input of a temporal filter parameter is used to temporally filter the texture data items. The temporally-filtered texture data items are applied to surfaces of the 3D spatial shared world model to generate a textured shared world model of the physical environment. The textured shared world model is then provided for display by a display device.

First claim

The invention claimed is:

1. A mixed reality experience sharing system for presenting a textured shared world model of a physical environment, the textured shared world model including a plurality of crowd-sourced structural data items that are geo-located at the physical environment, and a plurality of crowd-sourced texture data items that are geo-located at the physical environment, the mixed reality experience sharing system comprising:
a computing device comprising a processor; and
a mixed reality experience sharing program executed by the processor of the computing device, the mixed reality experience sharing program configured to:
receive the plurality of crowd-sourced structural data items including depth information;
stitch together the plurality of crowd-sourced structural data items to generate a 3D spatial shared world model of the physical environment;
receive the plurality of crowd-sourced texture data items from a plurality of sources, each of the texture data items comprising a time-stamped image or video, wherein the plurality of crowd-sourced texture data items comprise at least one hologram and metadata describing the hologram;
receive user input of a temporal filter parameter;
filter the plurality of crowd-sourced texture data items by the temporal filter parameter to generate a temporally-filtered subgroup of the crowd-sourced texture data items;
apply to surfaces of the 3D spatial shared world model of the physical environment the temporally-filtered subgroup of the crowd-sourced texture data items to generate the textured shared world model of the physical environment;
provide the textured shared world model of the physical environment to a display program for display by a display device; and
locate the hologram within the textured shared world model of the physical environment.

2. The mixed reality experience sharing system of claim 1 ,
wherein the temporal filter parameter corresponds to a predated time that is earlier than a current time.

3. The mixed reality experience sharing system of claim 2 , wherein the mixed reality experience sharing program is further configured to:
filter the plurality of crowd-sourced structural data items by the temporal filter parameter to generate a temporally-filtered subgroup of the crowd-sourced structural data items; and
stitch together the temporally-filtered subgroup of the crowd-sourced structural data items to generate the 3D spatial shared world model of the physical environment.

4. The mixed reality experience sharing system of claim 1 , wherein the mixed reality experience sharing program is further configured to receive the plurality of crowd-sourced texture data items from at least a user of the display device and a third party source.

5. The mixed reality experience sharing system of claim 1 , wherein each of the plurality of crowd-sourced texture data items is selected from the group consisting of light illuminating one or more of the crowd-sourced structural data items, an image of an object in the physical environment, a video of an object in the physical environment, and audio of the physical environment.

6. The mixed reality experience sharing system of claim 1 , wherein the mixed reality experience sharing program is further configured to prioritize the plurality of crowd-sourced structural data items according to one or more of a fidelity, an age, a trustworthiness of a source, and a social proximity of the source of the crowd-sourced structural data items.

7. The mixed reality experience sharing system of claim 1 , wherein the mixed reality experience sharing program is further configured to receive the plurality of crowd-sourced structural data items from at least one head-mounted display device.

8. The mixed reality experience sharing system of claim 1 , wherein the mixed reality experience sharing program is further configured to prioritize the plurality of crowd-sourced texture data items according to one or more of a trustworthiness of a source and a social proximity of the source of the crowd-sourced texture data items.

9. A method for presenting a textured shared world model of a physical environment via a display device, the method comprising:
receiving a plurality of crowd-sourced structural data items that are geo-located at the physical environment and include depth information;
stitching together the plurality of crowd-sourced structural data items to generate a 3D spatial shared world model of the physical environment;
receiving from a plurality of sources a plurality of crowd-sourced texture data items that are geo-located at the physical environment, each of the crowd-sourced texture data items comprising a time-stamped image or video, wherein the plurality of crowd-sourced texture data items comprise at least one hologram and metadata describing the hologram;
receiving user input of a temporal filter parameter;
filtering the plurality of crowd-sourced texture data items by the temporal filter parameter to generate a temporally-filtered subgroup of the crowd-sourced texture data items;
applying to surfaces of the 3D spatial shared world model of the physical environment the temporally-filtered subgroup of the crowd-sourced texture data items to generate the textured shared world model of the physical environment;
providing the textured shared world model of the physical environment to a display program for display by the display device; and
locating the hologram within the textured shared world model of the physical environment.

10. The method of claim 9 , wherein the temporal filter parameter corresponds to a predated time that is earlier than a current time.

11. The method of claim 10 , further comprising:
filtering the plurality of crowd-sourced structural data items by the temporal filter parameter to generate a temporally-filtered subgroup of the crowd-sourced structural data items; and
wherein stitching together the plurality of crowd-sourced structural data items comprises stitching together the temporally-filtered subgroup of the crowd-sourced structural data items to generate the 3D spatial shared world model of the physical environment.

12. The method of claim 9 , further comprising receiving the plurality of crowd-sourced texture data items from at least a user of the display device and a third party source.

13. The method of claim 9 , wherein each of the plurality of crowd-sourced texture data items are selected from the group consisting of light illuminating one or more of the crowd-sourced structural data items, an image of an object in the physical environment, a video of an object in the physical environment, and audio of the physical environment.

14. The method of claim 9 , further comprising prioritizing the plurality of crowd-sourced structural data items according to one or more of a fidelity, an age, a trustworthiness of a source, and a social proximity of the source of the crowd-sourced structural data items.

15. The method of claim 9 , further comprising prioritizing the plurality of crowd-sourced texture data items according to one or more of a trustworthiness of a source and a social proximity of the source of the crowd-sourced texture data items.

16. A mixed reality experience sharing system for presenting a textured shared world model of a physical environment, the textured shared world model including a plurality of crowd-sourced structural data items that are geo-located at the physical environment, and a plurality of crowd-sourced texture data items that are geo-located at the physical environment, the mixed reality experience sharing system comprising:
a

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Kamuda Nicholas
• Kinnebrew Peter Tobias

Classifications

• G06T2219/024

  Multi-user, collaborative environment · CPC title

• A63F13/213

  comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T15/04

  Texture mapping · CPC title

• G06F3/04815

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title