Dynamic detection of pause and resume for video communications

Patent metadata

Field	Value
Publication number	US-9232187-B2
Application number	US-201414296349-A
Country	US
Kind code	B2
Filing date	Jun 4, 2014
Priority date	Jun 4, 2014
Publication date	Jan 5, 2016
Grant date	Jan 5, 2016

Abstract

Computing devices may implement dynamic detection of pause and resume for video communications. Video communication data may be capture at a participant device in a video communication. The video communication data may be evaluated to detect a pause or resume event for the transmission of the video communication data. Various types of video, audio, and other sensor analysis may be used to detect when a pause event or a resume event may be triggered. For triggered pause events, at least some of the video communication data may no longer be transmitted as part of the video communication. For triggered resume events, a pause state may cease and all of the video communication data may be transmitted.

First claim

What is claimed is:

1. A system, comprising:
one or more audio sensors, configured to capture audio data;
one or more video sensors, configured to capture image data;
one or more processors;
a memory comprising program instructions that when executed by the one or more processors cause the one or more processors to implement:
a video communication module, configured to:
receive a stream of video communication data for a current video communication from at least one of the audio sensors and at least one of the video sensors at a transmitting participant for transmission to one or more recipient participants of the current video communication, wherein the current video communication is a peer-managed connection between the transmitting participant and the one or more recipient participants;
dynamically evaluate the stream of video communication data to detect a pause event for the transmitting participant of the current video communication; and
in response to detection of the pause event, pause the current video communication for the transmitting participant such that at least some of the stream of video communication data is not transmitted from the transmitting participant to the one or more recipient participants.

2. The system of claim 1 , wherein to dynamically evaluate the stream of video communication data, the video communication module is configured to:
perform at least one of video analysis or audio analysis in order to determine an active exchange indicator for the transmitting participant; and
determine that the active exchange indicator is below an exchange threshold for the transmitting participant in order to trigger the pause event.

3. The system of claim 1 , wherein the dynamic evaluation of the stream of video communication data is based on identification of the at least one audio sensor or the at least one video sensor from which a portion of the video communication data is obtained.

4. The system of claim 1 ,
wherein the program instructions cause the one or more processors to implement a user interface module configured to detect a selection of a resume user interface element or a pause user interface element;
wherein the video communication module is further configured to:
receive an indication of a selection of the resume user interface element;
in response to receipt of the selection, resume the current video communication such that at least a portion of the stream of video communication data is transmitted again to the one or more recipient participants.

5. The system of claim 1 , wherein the video communication module is further configured to:
dynamically evaluate the stream of video communication data to detect a resume event for the transmitting participant of the current video communication; and
in response to detection of the resume event, resume the current video communication such that all of the stream of video communication data is transmitted to the one or more recipient participants.

6. The system of claim 1 , wherein the system is a mobile phone.

7. A method, comprising:
performing, by one or more computing devices:
capturing a stream of video communication data for a transmitting participant in a current video communication for transmission to one or more recipient participants of the current video communication, wherein the current video communication is a peer-managed connection between the transmitting participant and the one or more recipient participants;
dynamically evaluating, at the transmitting participant, the stream of video communication data to detect a pause event for the transmitting participant of the current video communication; and
in response to detecting the pause event, pausing the current video communication for the transmitting participant such that at least some of the stream of video communication data is not transmitted from the transmitting participant to the one or more recipient participants.

8. The method of claim 7 , wherein said dynamically evaluating the stream of video communication data to detect the pause event comprises:
performing at least one of video analysis or audio analysis in order to determine an active exchange indicator for the transmitting participant; and
determining that the active exchange indicator is below an exchange threshold for the transmitting participant in order to trigger the pause event.

9. The method of claim 8 , wherein said performing at least one of the video analysis or the audio analysis in order to determine the active exchange indicator for the transmitting participant is based on a particular capture source of a portion of the stream of video communication data.

10. The method of claim 7 , further comprising:
dynamically evaluating, at the transmitting participant, the stream of video communication data to detect a resume event for the transmitting participant of the current video communication; and
in response to detecting the resume event, resuming the current video communication such that at least a portion of the stream of video communication data is transmitted again to the one or more recipient participants.

11. The method of claim 7 , wherein the portion of the stream of video communication data transmitted again to one of the one or more recipient participants is than the portion of the stream of video data transmitted again to another one of the one or more recipient participants.

12. The method of claim 7 , wherein the peer-managed connection between the transmitting participant and the one or more recipient participants is maintained in response to detecting the pause event.

13. The method of claim 7 , wherein the one or more recipient participants are a plurality of recipient participants, and wherein the current video communication is a multi-way video communication among the plurality of recipient participants and the transmitting participant.

14. A non-transitory, computer-readable storage medium, storing program instructions that when executed by a mobile computing device causes the mobile computing device to implement:
capturing a stream of video communication data for a transmitting participant in a current video communication for transmission to one or more recipient participants of the current video communication, wherein the current video communication is a peer-managed connection between the transmitting participant and the one or more recipient participants, wherein the transmitting participant is in a paused state such that at least some of the stream of video communication data is not transmitted to the one or more recipient participants;
dynamically evaluating the stream of video communication data to detect a resume event for the transmitting participant of the current video communication; and
in response to detecting the resume event, resuming transmission of at least a portion of the stream of current video communication data to the one or more recipient participants.

15. The non-transitory, computer-readable storage medium of claim 14 , wherein said dynamically evaluating the video communication data to detect the resume event comprises:
performing at least one of video analysis or audio analysis in order to determine an active exchange indicator for the transmitting participant; and
determining that the active exchange indicator is above an exchange threshold for the transmitting participant in order to trigger the resume event.

16. The non-transitory, computer-readable storage medium of claim 14 , wherein the program instructions cause the mobile computing device to further implement:
prior to the transmitting participan

Assignees

• Apple Inc

Inventors

• Zhou Xiaosong
• Wu Hsi-Jung
• Chung Chris Y
• Normile James O
• Abuan Joe S
• Jeong Hyeonkuk
• Yang Yan
• Johar Gobind
• Jansen Thomas

Classifications

• H04L12/1827

  Network arrangements for conference optimisation or adaptation · CPC title

• H04L12/1822

  Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission · CPC title

• H04L51/02

  using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages · CPC title

• H04N7/15 Primary

  Conference systems · CPC title

• H04N7/147

  Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q) · CPC title