Presentation of enhanced communication between remote participants using augmented and virtual reality

Patent metadata

Field	Value
Publication number	US-9959676-B2
Application number	US-201514601535-A
Country	US
Kind code	B2
Filing date	Jan 21, 2015
Priority date	Jan 24, 2014
Publication date	May 1, 2018
Grant date	May 1, 2018

Abstract

Embodiments disclosed herein provide methods, systems, and computer readable storage media for presenting a meeting between remote participants. In a particular embodiment, a method provides, during a conference session, compiling view information used to determine a first view seen by the first participant. The method further provides receiving conference video comprising at least video captured of a second participant remote to the physical location of the first participant. Also, the method provides presenting the video captured of the second participant such that the second participant appears at a first position of a meeting location in the first view to the first participant.

First claim

What is claimed is:

1. A non-transitory computer readable storage medium having instructions stored thereon for presenting a conference session established by a conferencing system to facilitate a meeting between remote participants that each use Augmented Reality (AR) or Virtual Reality (VR) to participate in the meeting, the instructions, when executed by a head mounted display (HMD) worn by a first participant of the remote participants, direct the HMD to:
during the conference session, compile view information used to determine a first view seen by the first participant located at a meeting location, wherein the first view includes what the first participant sees of the meeting location through transparent lenses of the HMD;
receive, via a communication interface of the HMD, a conference video stream comprising at least video captured of a second participant, of the remote participants, remote to the meeting location; and
present the video captured of the second participant on the transparent lenses such that the second participant appears at a first position of the meeting location in the first view to the first participant to appear as though the second participant is physically at the meeting location with the first participant.

2. The non-transitory computer readable medium of claim 1 , wherein the instructions further direct the HMD to:
transfer the view information to a conferencing server; and
wherein the conferencing server uses the view information to process the video captured of the second participant to account for a viewing angle of the first view relative to the first position of the meeting location.

3. The non-transitory computer readable medium of claim 1 , wherein the instructions further direct the HMD to:
use the view information to process the video captured of the second participant to account for a viewing angle of the first view relative to the first position of the meeting location.

4. The non-transitory computer readable medium of claim 1 , wherein the instructions further direct the HMD to:
determine where the first position is located within the meeting location.

5. The non-transitory computer readable medium of claim 1 , wherein the conference video stream further comprises video captured of a third participant remote to the physical location of the first participant, and the instructions further direct the HMD to:
present the video captured of the third participant such that the third participant appears at a second position of the meeting location in the first view to the first participant.

6. The non-transitory computer readable medium of claim 1 , wherein the video captured of the second participant includes audio captured of the second participant, and the instructions further direct the HMD to:
present the audio captured of the second participant such that the audio captured of the second participant appears to be coming from the first position of the meeting location.

7. The non-transitory computer readable medium of claim 1 , wherein to compile the view information the instructions direct the HMD to:
capture video of a physical scene outward from the front of the HMD.

8. The non-transitory computer readable medium of claim 1 , wherein the instructions further direct the HMD to:
receive information from a conferencing server that indicates the first position of the meeting location.

9. A head mounted display (HMD) for presenting a conference session established by a conferencing system to facilitate a meeting between remote participants that each use Augmented Reality (AR) or Virtual Reality (VR) to participate in the meeting, the HMD comprising:
a processing system configured to, during a conference session for the meeting, compile view information used to determine a first view seen by a first participant of the AR and VR participants wearing the HMD and located at a meeting location, wherein the first view includes what the first participant sees of the meeting location through transparent lenses of the HMD;
a communication interface configured to receive a conference video stream comprising at least video captured of a second participant, of the AR and VR participants, remote to the meeting location; and
a display system configured to present the video captured of the second participant on the transparent lenses such that the second participant appears at a first position of the meeting location in the first view to the first participant to appear as though the second participant is physically at the meeting location with the first participant.

10. The HMD of claim 9 , further comprising:
the communication interface configured to transfer the view information to a conferencing server; and
wherein the conferencing server uses the view information to process the video captured of the second participant to account for a viewing angle of the first view relative to the first position of the meeting location.

11. The HMD of claim 9 , further comprising:
the processing system configured to use the view information to process the video captured of the second participant to account for a viewing angle of the first view relative to the first position of the meeting location.

12. The HMD of claim 9 , further comprising:
the processing system configured to determine where the first position is located within the meeting location.

13. The HMD of claim 9 , wherein the conference video stream further comprises video captured of a third participant remote to the physical location of the first participant, and the HMD further comprising:
the display system configured to present the video captured of the third participant such that the third participant appears at a second position of the meeting location in the first view to the first participant.

14. The HMD of claim 9 , wherein the video captured of the second participant includes audio captured of the second participant, and the HMD further comprising:
an audio system configured to present the audio captured of the second participant such that the audio captured of the second participant appears to be coming from the first position of the meeting location.

15. The HMD of claim 9 , further comprising:
a video capture system configured to capture video of a physical scene outward from the front of the HMD; and
wherein the processing system configured to compile the view information comprises the processing system configured to receive the captured video of the physical scene outward from the front of the HMD.

16. The HMD of claim 9 , further comprising:
the communication interface configured to receive information from a conferencing server that indicates the first position of the meeting location.

Assignees

• Avaya Inc

Inventors

• Barzuza Tamar
• Wiener Yair
• Modai Ori

Classifications

• H04N7/15

  Conference systems · CPC title

• H04L65/403

  Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• H04N13/0429

  Electricity · mapped topic

• H04N7/157 Primary

  defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827) · CPC title