Techniques for enhancing user experience in video conferencing

Patent metadata

Field	Value
Publication number	US-2017280098-A1
Application number	US-201415504967-A
Country	US
Kind code	A1
Filing date	Sep 26, 2014
Priority date	Sep 26, 2014
Publication date	Sep 28, 2017
Grant date	—

Abstract

Techniques are disclosed for enhancing user experience in video conferencing. In accordance with some embodiments, the graphical user interface (GUI) displayed on a device involved in a video conferencing session may undergo dynamic adjustment of its video composition, for example, to render video content in either a prominent or a thumbnail region of the GUI. Reorganization of the GUI's video composition may be performed, for example: (1) automatically based on detected audio activity levels of the video conferencing participants; and/or (2) upon user instruction. In accordance with some embodiments, individualized volume control over video conferencing participants may be provided. In accordance with some embodiments, the resolution and/or frame rate of video data captured at a source device involved in a video conferencing session may be adaptively varied, for example, during capture and/or processing before encoding based on the detected audio activity level of the user of that source device.

First claim

1 . A non-transitory computer program product encoded with instructions that, when executed by one or more processors, causes a process to be carried out, the process comprising:
receiving audio data in a video conferencing session; analyzing the audio data to determine an audio activity level of at least one participant of the video conferencing session; and adjusting a video composition of a graphical user interface (GUI) based on the audio activity level of the at least one participant.

2 . The non-transitory computer program product of claim 1 , wherein analyzing the audio data to determine the audio activity level of the at least one participant comprises:
sampling the audio data received in the video conferencing session and computing therefrom an audio signature to identify which participant is associated with the audio data; and comparing the audio data against an audio threshold.

3 . The non-transitory computer program product of claim 1 , wherein upon comparing the audio data against the audio threshold, if the audio data exceeds the audio threshold, then adjusting the video composition of the GUI comprises:
automatically transitioning presentation of a video stream representative of the participant from a thumbnail region of the GUI to a prominent region of the GUI; automatically transitioning presentation of a video stream representative of the participant from a thumbnail region of the GUI to a prominent region of the GUI and automatically transitioning presentation of a video stream representative of another participant from the prominent region of the GUI to the thumbnail region of the GUI; or maintaining presentation of a video stream representative of the participant within a prominent region of the GUI.

4 . The non-transitory computer program product of claim 1 , wherein upon comparing the audio data against the audio threshold, if the audio data does not exceed the audio threshold, then adjusting the video composition of the GUI comprises:
automatically transitioning presentation of a video stream representative of the participant from a prominent region of the GUI to a thumbnail region of the GUI; or maintaining presentation of a video stream representative of the participant within a thumbnail region of the GUI.

5 . The non-transitory computer program product of claim 1 , wherein adjusting the video composition of the GUI comprises at least one of:
transitioning presentation of a video stream representative of at least one of a remote participant and an object/scene of interest between a prominent region of the GUI and a thumbnail region of the GUI; adjusting a resolution of a video stream representative of at least one remote participant; and adjusting a frame rate of a video stream representative of at least one remote participant.

6 . The non-transitory computer program product of claim 1 , wherein adjusting the video composition of the GUI is performed automatically based on the audio activity level of a local or remote participant causing the adjusting.

7 . The non-transitory computer program product of claim 1 , wherein adjusting the video composition of the GUI is further based on input received via a touch-sensitive display on which the GUI is presented.

8 . The non-transitory computer program product of claim 1 , wherein at least a portion of the process is carried out via at least one of an IR.94-based implementation and a WebRTC-based implementation.

9 . A non-transitory computer program product encoded with instructions that, when executed by one or more processors, causes a process to be carried out, the process comprising:
receiving audio data in a video conferencing session, the audio data including at least one audio stream associated with an individual remote video conferencing participant; and adjusting a volume level of the at least one audio stream associated with the individual remote video conferencing participant.

10 . The non-transitory computer program product of claim 9 , wherein at least a portion of the process is carried out via a WebRTC-based implementation.

11 . The non-transitory computer program product of claim 9 , wherein:
prior to adjusting the volume level of the at least one audio stream associated with the individual remote video conferencing participant, the process further comprises splitting the audio data into a plurality of audio streams, the plurality including the at least one audio stream associated with the individual remote video conferencing participant; and after adjusting the volume level of the at least one audio stream associated with the individual remote video conferencing participant, the process further comprises re-synthesizing the plurality of audio streams into a single audio stream.

12 . The non-transitory computer program product of claim 11 , wherein at least a portion of the process is carried out via an IR.94-based implementation.

13 . A non-transitory computer program product encoded with instructions that, when executed by one or more processors, causes a process to be carried out, the process comprising:
receiving audio data in a video conferencing session; analyzing the audio data to determine therefrom an audio activity level of a local participant of the video conferencing session; and adjusting at least one of a resolution and a frame rate of video data transmitted in the video conferencing session based on the audio activity level of the local participant.

14 . The non-transitory computer program product of claim 13 , wherein adjusting at least one of the resolution and the frame rate of the video data transmitted in the video conferencing session comprises:
adjusting at least one of a capture resolution and a capture frame rate of an image capture device configured to capture the video data before encoding thereof.

15 . The non-transitory computer program product of claim 13 , wherein adjusting at least one of the resolution and the frame rate of the video data transmitted in the video conferencing session comprises:
scaling at least one of the resolution and the frame rate of captured video data before encoding thereof.

16 . The non-transitory computer program product of claim 13 , wherein analyzing the audio data to determine therefrom the audio activity level of the local participant comprises:
sampling the audio data received in the video conferencing session and computing therefrom an audio signature to identify which participant is associated with the audio data; and comparing the audio data against an audio threshold.

17 . The non-transitory computer program product of claim 16 , wherein upon comparing the audio data against the audio threshold, if the audio data exceeds the audio threshold, then adjusting at least one of the resolution and the frame rate of the video data comprises at least one of:
automatically increasing at least one of a capture resolution and a capture frame rate of an image capture device configured to capture the video data before encoding thereof; and automatically upscaling at least one of the resolution and the frame rate of the video data before encoding thereof.

18 . The non-transitory computer program product of claim 16 , wherein upon comparing the audio data against the audio threshold, if the audio data does not exceed the audio threshold, then adjusting at least one of the resolution and the frame rate of the video data comprises at least one of:
automatically decreasing at least one of a capture resolution and a capture frame rate of an image capture device configured

Assignees

• Intel Corp

Inventors

• Sethuraman Ramanathan
• Bn Raghunandan
• Bhaskar Rajesh
• Giacalone Jean-Pierre

Classifications

• H04N7/15 Primary

  Conference systems · CPC title

• H04L12/1827

  Network arrangements for conference optimisation or adaptation · CPC title

• G01H3/14

  Measuring mean amplitude; Measuring mean power; Measuring time integral of power · CPC title