Optimizing panoramic image composition

Patent metadata

Field	Value
Publication number	US-10015446-B2
Application number	US-201615252085-A
Country	US
Kind code	B2
Filing date	Aug 30, 2016
Priority date	Oct 5, 2015
Publication date	Jul 3, 2018
Grant date	Jul 3, 2018

Abstract

An automatic process for producing professional, directed, production crew quality, video for videoconferencing is described. The process includes a method for positioning a 360 degree panoramic strip view of a room. The method includes receiving motion detection data, accepting presence sensor data, receiving video data from cameras related to a room view, and centering individuals in a room view, based on the received motion detection data, the accepted presence sensor data and received video data. The method also includes zooming onto the centered individuals in the room view, obtaining sound source localization data and active talker information, and determining how to arrange the display of individuals in a logical manner that is visually pleasing and aides understanding. Rule based logic is applied to assist with the automatic processing of video into director quality video production. Various video sources may be used as video stream sources for the automated system.

First claim

What is claimed is:

1. A method for positioning a 360 degree panoramic strip view of a room, the method comprising:
receiving, at a processor, motion detection data;
accepting, at the processor, presence sensor data;
receiving, at the processor, video data from cameras related to a room view;
centering, under control of the processor, individuals in a room view, using the received motion detection data, accepted presence sensor data and received video data;
zooming onto the centered individuals of the room view in one or more zoomed views;
obtaining sound source localization data and active talker information about an active talker;
increasing a level of magnification of at least one of the zoomed views so as to remove at least one empty seat from one or more of the zoomed views;
determining at the processor, using the obtained sound source localization data, the obtained active talker information and the received motion detection data, where, within a panoramic strip in a video layout, to place the zoomed views of individuals in the room; and
determining, at the processor, where to place the panoramic strip within the video layout, so as to prevent the panoramic strip overlaying display of the active talker in an active talker video stream, while maintaining the scale of the active talker video stream.

2. The method of claim 1 , further comprising placing the panoramic strip in one of three locations, a top of the video layout, a bottom of the video layout, or hidden.

3. The method of claim 1 , further comprising placing the zoomed views in the center of the panoramic strip.

4. The method of claim 3 , wherein placing the zoomed views in the center of the panoramic strip comprises left-right balancing of individuals in the room view.

5. The method of claim 1 , wherein two or more panoramic strips are positioned in a stacked configuration, and the zoomed views are placed in the center of the strips.

6. The method of claim 1 , further comprising placing the zoomed views in the panoramic strip such that no views which do not contain individuals are included in the panoramic strip.

7. The method of claim 1 , further comprising placing the zoomed views in the panoramic strip such that views which do not contain individuals are not placed between views containing individuals.

8. The method claim 1 , further comprising continuously transmitting the panoramic strip in hidden video lines.

9. A video production system for a room, the video production system comprising:
a memory for storing program code for execution; and
a processor which executes stored program code and is configured to:
receive motion detection data;
accept presence sensor data;
receive information regarding video recorded by cameras in a room;
center, using the received motion detection data, accepted presence sensor data and received information, individuals in a room view;
zoom onto the centered individuals of the room view in one or more zoomed views;
obtain sound source localization data and active talker information corresponding to at least one active talker;
determine, using the obtained sound source localization data, the obtained active talker information and the received motion detection data, where, within a panoramic strip in a video layout, to place the zoomed views of individuals in the room; and
place the panoramic strip within the video layout, so as to prevent the panoramic strip overlaying display of the active talker, while maintaining the scale of the display of the active talker within the video layout.

10. The video production system of claim 9 , wherein the processor is further configured to place the panoramic strip in one of three locations, a top of the video layout, a bottom of the video layout, or hidden.

11. The video production system of claim 9 , wherein the processor is further configured to place the zoomed views in the center of the panoramic strip.

12. The video production system of claim 11 , wherein placing the zoomed views in the center of the panoramic strip comprises left-right balancing of individuals in the room view.

13. The video production system of claim 9 , wherein the processor is further configured to position two or more panoramic strips in a stacked configuration in which the zoomed views are placed in the center of the strips.

14. The video production system of claim 9 , wherein the processor is further configured to place the zoomed views in the panoramic strip such that no views which do not contain individuals are included in the panoramic strip.

15. The video production system of claim 9 , wherein the processor is further configured to place the zoomed views in the panoramic strip such that views which do not contain individuals are not placed between views containing individuals.

16. The video production system of claim 9 , wherein the processor is further configured to transmit the strip in hidden video lines.

17. The method of claim 1 , further comprising detecting two active talkers within the room and placing zoom views of the two active talkers in the panoramic strip such that the zoom views containing the two active talkers are adjacent, and the faces of the two active talkers point towards one another.

18. A non-transitory storage medium storing instructions executable by a processor, the instructions comprising instructions to:
receive motion detection data;
accept presence sensor data;
receive video information from one or more cameras;
center, using the received motion detection data, accepted presence sensor data and received video information, individuals in a room view using at least one display device;
zoom onto the centered individuals of the room view in one or more zoomed views;
obtain sound source localization data and active talker information corresponding to at least one active talker;
determine, based, at least in part, on the obtained sound source localization data, the obtained active talker information and the received motion detection data, where, within a panoramic strip in a video layout of the display device, to place the zoomed views of the individuals; and
place the panoramic strip within the video layout, so as to prevent the panoramic strip overlaying display of the active talker, while maintaining the scale of the display of the active talker within the video layout.

19. The non-transitory storage medium of claim 18 , wherein the instructions further comprise instructions to place the panoramic strip in one of three locations, a top of the video layout, a bottom of the video layout, or hidden.

20. The non-transitory storage medium of claim 19 , wherein the instructions further comprise instructions to place the zoomed views in the center of the panoramic strip.

Assignees

• Polycom Inc

Inventors

• Nimri Alain Elon
• Gao Shu
• Schaefer Stephen
• Murphy Robert

Classifications

• G06T2207/30196

  Human being; Person · CPC title

• H04N7/147

  Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q) · CPC title

• G10L21/0272

  Voice signal separating · CPC title

• H04N7/152 Primary

  Multipoint control units therefor · CPC title

• G10L25/72

  for transmitting results of analysis · CPC title