Providing a tele-immersive experience using a mirror metaphor

Patent metadata

Field	Value
Publication number	US-9641805-B2
Application number	US-201615073928-A
Country	US
Kind code	B2
Filing date	Mar 18, 2016
Priority date	Feb 20, 2013
Publication date	May 2, 2017
Grant date	May 2, 2017

Abstract

A tele-immersive environment is described that provides interaction among participants of a tele-immersive session. The environment includes two or more set-ups, each associated with a participant. Each set-up, in turn, includes mirror functionality for presenting a three-dimensional virtual space for viewing by a local participant. The virtual space shows at least some of the participants as if the participants were physically present at a same location and looking into a mirror. The mirror functionality can be implemented as a combination of a semi-transparent mirror and a display device, or just a display device acting alone. According to another feature, the environment may present a virtual object in a manner that allows any of the participants of the tele-immersive session to interact with the virtual object.

First claim

What is claimed is:

1. A method comprising:
receiving local camera information that represents a local participant associated with a local system;
receiving remote input information that represents at least one remote participant from at least one remote system operating in a geographical location that is different from the local system;
composing scene information based at least in part on the local camera information and the remote input information;
providing a presentation of a virtual space based at least in part on the scene information, the virtual space showing the local participant and the at least one remote participant as if the local participant and the at least one remote participant were physically present at a same location; and
omitting a virtual image of the local participant from the virtual space in response to receiving an instruction from the local participant indicating not to display the virtual image of the local participant in the virtual space.

2. The method of claim 1 , further comprising transferring local input information to the at least one remote system, the local input information corresponding to information associated with objects identified by the local system.

3. The method of claim 2 , wherein the local input information comprises one or more of the local camera information or local three-dimensional object information that provides a three-dimensional representation of one or more objects in a real space that is local to the local system.

4. The method of claim 1 , further comprising providing a functionality to enable the local participant to interact with a virtual object presented in the virtual space.

5. The method of claim 1 , further comprising:
tracking a location or movement of the local participant in a real space of the local system; and
transferring the tracked location or movement of the local participant to the at least one remote system.

6. An apparatus comprising:
a local processing system to:
receive remote input information that represents a plurality of remote participants from a plurality of remote systems operating in one or more geographical locations different from each other; and
compose scene information based at least in part on the remote input information; and

a mirror functionality to present a virtual space based at least in part on the scene information, the virtual space showing the plurality of remote participants as if the plurality of remote participants were physically present at a same location without displaying a virtual image of a local participant associated with the local processing system.

7. The apparatus of claim 6 , further comprising an image capture functionality to capture local camera information that represents the local participant.

8. The apparatus of claim 7 , wherein the local processing system is further configured to generate local object information based at least in part on the local camera information, and wherein the local processing system composes the scene information based further on the local object information.

9. The apparatus of claim 8 , wherein the virtual space further shows the local participant as if the local participant and the plurality of remote participants were physically present the same location in response to receiving an instruction from the local participant to display the virtual image of the local participant in the virtual space.

10. The apparatus of claim 6 , wherein the local processing system is further configured to transfer local input information to at least one of the plurality of remote systems, the local input information corresponding to information associated with one or more objects identified by the local processing system.

11. The apparatus of claim 6 , wherein the mirror functionality causes a display device to receive the scene information and present respective virtual images of the plurality of remote participants.

12. The apparatus of claim 6 , wherein the local processing system includes functionality that enables the local participant to interact with one or more virtual-reflected objects and one or more virtual-actual objects, a virtual-reflected object of the one or more virtual-reflected objects corresponding to an object projected from a perspective of reflections on a mirror surface and a virtual-actual object of the one or more virtual-actual objects corresponding to an object projected from a perspective of an entity that is placed before the mirror surface.

13. The apparatus of claim 6 , further comprising a management module that receives information specified by the local participant and provides participant-specified information, wherein the local processing system is further configured to present the participant-specified information using the mirror functionality.

14. The apparatus of claim 6 , further comprising a management module that provides a control feature and manages one or more interactions with the control feature, wherein the local processing system is further configured to present the control feature using the mirror functionality.

15. A local system comprising:
an image capture functionality to receive local camera information that represents a local participant associated with the local system;
a local processing system to:
receive remote input information that represents at least one remote participant from at least one remote system operating in a geographical location that is different from the local system; and
compose scene information based at least in part on the local camera information and the remote input information; and

a mirror functionality to:
provide a presentation of a virtual space based at least in part on the scene information, the virtual space showing the local participant and the at least one remote participant as if the local participant and the at least one remote participant were physically present at a same location; and
omit a virtual image of the local participant from the virtual space based on an instruction from the local participant indicating not to display the virtual image of the local participant in the virtual space.

16. The local system of claim 15 , wherein the local processing system is further configured to transfer local input information to the at least one remote system, the local input information corresponding to information associated with one or more objects identified by the local processing system.

17. The local system of claim 15 , wherein the local processing system includes functionality that enables the local participant to interact with one or more virtual-reflected objects and one or more virtual-actual objects, a virtual-reflected object of the one or more virtual-reflected objects corresponding to an object projected from a perspective of reflections on a mirror surface and a virtual-actual object of the one or more virtual-actual objects corresponding to an object projected from a perspective of an entity that is placed before the mirror surface.

18. The local system of claim 15 , further comprising a management module that receives information specified by the local participant and provides participant-specified information, wherein the local processing system is further configured to present the participant-specified information using the mirror functionality.

19. The local system of claim 15 , further comprising a management module that provides a control feature and manages one or more interactions with the control feature, wherein the local processing system is further configured to prese

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Wilson Andrew D
• Zhang Zhengyou
• Chou Philip A
• Fishman Neil S
• Gillett Donald M
• Benko Hrvoje

Classifications

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F2203/0381

  Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• A47G1/02

  Mirrors used as equipment (hand, pocket, or shaving mirrors A45D42/00; {optical viewing means for vehicles B60R1/00}) · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title