Dynamic viewing perspective of remote scenes

Patent metadata

Field	Value
Publication number	US-9787939-B1
Application number	US-201414547892-A
Country	US
Kind code	B1
Filing date	Nov 19, 2014
Priority date	Nov 19, 2014
Publication date	Oct 10, 2017
Grant date	Oct 10, 2017

Abstract

Video conferencing is performed using near-end and far-end devices at near-end and far-end sites, respectively. Each device has a display surface and a video camera. The view of the far-end scene displayed by the near-end device is selected as a function of the position of the near-end user relative to the near-end device. As the near-end user moves toward the left, the view pans toward the right of the far-end scene. As the near-end user moves toward the right, the view pans toward the left of the far-end scene.

First claim

What is claimed is:

1. A first device, comprising:
a video camera configured to operate from a first site to produce first video of a first scene, wherein the first video comprises a first sequence of video frames;
a display surface having a lateral center, the display surface having a right and a left from a perspective of a user facing the display surface;
operating logic configured to perform acts comprising:
providing the first video to a second device for display by the second device,
wherein the second device is at a second site;
receiving second video of a second scene from the second device, wherein the second video comprises a second sequence of video frames;
analyzing a first video frame of the first video to detect a user who is in the first scene, wherein the user is at a viewing angle relative to the lateral center of the display surface, the viewing angle being defined by a first ray and a second ray, the first ray extending from the lateral center of the display surface to the user, the second ray extending orthogonally from the lateral center of the display surface;
extracting a region of each video frame of the second video based at least in part on the viewing angle to create an extracted region for each video frame;
analyzing a second video frame of the first video to detect a change in the viewing angle;
based at least in part on analyzing the second video frame, generating third video, the generating comprising at least one of:
determining to position each extracted region for each frame at leftward regions within each video frame in response to the change in the viewing angle being toward the right of the display surface; or
determining to position each extracted region for each frame at rightward regions within each video frame in response to the change in the viewing angle being toward the left of the display surface; and

causing the third video to be displayed on the display surface.

2. The first device of claim 1 , further comprising continuously moving each extracted region within each video frame in response to movement of the user relative to the display surface.

3. The first device of claim 1 , wherein each extracted region has a size, the acts further comprising changing the size of each extracted region as a function of a change in a distance of the user from the display surface.

4. The first device of claim 1 , wherein:
the region is at a vertical position within each video frame;
the display surface has a top and a bottom; and
the acts further comprise (a) moving each extracted region for each frame upward relative to the region in each video frame in response to a change in the viewing angle toward the bottom of the display surface and (b) moving each extracted region for each frame downward relative to the region in each video frame in response to a change in the viewing angle toward the top of the display surface.

5. A device for operation at a first site, comprising:
a display;
a user tracking system configured to detect a position of a user relative to the display;
operating logic configured to perform acts comprising:
receiving video that represents a scene of an environment of a second site;
causing the video that represents the scene to be presented on the display;
determining, based at least in part on the position, an area of the scene corresponding to a viewing angle of the user, the area of the scene corresponding to a portion of the video presented at a first region of the display;
extracting the portion from the video to create an extracted portion of the video;
detecting a change in the position of the user relative to the display;
based at least in part on determining the change in the position comprises a rightward change in the position from a perspective of the user:
determining a second region of the display leftward relative to the first region from the perspective of the user; and
causing the extracted portion to be displayed at the second region of the display; and

based at least in part on determining the change in the position comprises a leftward change in the position from the perspective of the user:
determining a third region of the display rightward relative to the first region from the perspective of the user; and
causing the extracted portion to be displayed at the third region of the display.

6. The device of claim 5 , wherein the position comprises one or more of:
a lateral position of the user relative to the display;
a distance of the user from the display; or
an angular position of the user relative to the display.

7. The device of claim 5 , the acts further comprising causing the extracted portion of the video to be presented at a fourth region positioned vertically relative to the first region of the display in response to a vertical change in the position.

8. The device of claim 5 , the portion of the video having a size on the display, the acts further comprising changing the size of the portion of the video in response to a change in the position toward or away from the display.

9. The device of claim 5 , the acts further comprising cropping the received video to create the extracted portion of the video.

10. The device of claim 5 , the acts further comprising controlling a camera that is at the second site to move the area leftward and rightward.

11. The device of claim 5 , the acts further comprising selecting from multiple cameras at the second site to cause the extracted portion of the scene to be displayed at the second region or the third region.

12. The device of claim 5 , wherein the received video includes multiple video streams produced respectively by multiple cameras at the second site, the multiple cameras having respectively different perspectives of the scene, the acts further comprising selecting the first region from each of two or more of the multiple video streams to determine the portion of the video.

13. The device of claim 5 , the acts further comprising causing the first region of the received video to be received at a higher resolution or frame rate than a second region of the received video, wherein the first region corresponds to the portion.

14. A method, comprising:
detecting, by one or more sensors of a computing device, a first position of a user relative to a display surface, wherein the display surface is at a first site;
receiving video that represents a scene of an environment of a second site;
causing the video that represents the scene of the environment of the second site to be presented on the display surface;
determining, based at least in part on the first position, an area of the scene corresponding to a first viewing angle of the user;
identifying a portion of the video corresponding to the area of the scene, the portion of the video being presented at a first region of the display surface;

detecting, by the one or more sensors, a change in position of the user from the first position to a second position relative to the display surface;
extracting the portion of the video from the video to create an extracted portion of the video;
determining, based at least in part on the second position, a second viewing angle of the user;
determining, based at least in part on the second viewing angle, a second region of the display surface at which to place the extracted portion of the video,

wherein determining the second region comprises at least one of:
determining that the second region is leftward relative to the first region in response to detecting a rightward change in the position relative to the display surface; or
determining that the second region is rightw

Assignees

• Amazon Tech Inc

Inventors

• Beguin Julien George
• Breitbard Eric Alan
• Murdock Robert Steven
• Li Meng
• Mcqueen Michael Douglas

Classifications

• H04N7/141 Primary

  between two video terminals, e.g. videophone (telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085) · CPC title

• H04N7/147 Primary

  Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q) · CPC title