System and method for operating an intelligent face framing management system for videoconferencing applications

Patent metadata

Field	Value
Publication number	US-11463270-B2
Application number	US-202117161582-A
Country	US
Kind code	B2
Filing date	Jan 28, 2021
Priority date	Jan 28, 2021
Publication date	Oct 4, 2022
Grant date	Oct 4, 2022

Abstract

A system or method executing an intelligent face framing management system comprising a processor to execute code instructions of a multimedia multi-user collaboration application to join a videoconference session, a display screen, a speaker, a video camera, and a microphone where the video camera captures a videoframe of a user and the processor to input videoframe data, including the detected user's image, into a trained neural network to determine image features for the intelligent face framing management system to generate optimized face framing adjustments center or normalize the user's image in the captured videoframes or intelligently select an alternate camera and prepare those videoframes for transmission.

First claim

What is claimed is:

1. An information handling system executing an intelligent face framing management system comprising:
a processor configured to execute code instructions of a multimedia multi-user collaboration application to join a videoconference session with a remotely located computing device;
a display screen, a speaker, and a microphone;
a video camera configured to capture a videoframe of a user's image;
the processor to input the videoframe data, including the user's image, and audiovisual (AV) processing instructions to be applied to videoframes into a trained neural network of the intelligent face framing management system;
the processor to execute code instructions of the trained neural network of the intelligent face framing management system to determine a face center and a face image;
the intelligent face framing management system to output an optimized normalizing face framing adjustments to adjust the size of the face image relative to the captured videoframe to within a size percentage range threshold of the face image in the captured videoframe;
the processor to execute an AV processing instruction module configured to execute a digital zoom process on the face image in the captured videoframe to meet the size percentage range threshold in the videoframe; and
a network interface device configured to transmit a processed, encoded media sample, including the captured videoframe to a remotely located computing device participating in the videoconference session.

2. The information handling system of claim 1 further comprising:
the processor to detect user movement with a proximity sensor, during additional captured videoframes;
the processor to execute the intelligent face framing management system to output an adjusted optimized normalizing face framing adjustment in response to the detected user movement.

3. The information handling system of claim 1 further comprising:
the processor to detect user movement toward or away from the video camera in additional captured videoframes;
the processor to the additional captured videoframes, including the user's image, into the trained neural network of the intelligent face framing management system; and
the processor to execute the AV processing instruction module configured to execute the digital zoom process on the face image in the additional captured videoframes to meet the size percentage range threshold in the videoframe according to a second optimized normalizing face framing adjustment.

4. The information handling system of claim 1 further comprising:
the processor to execute code instructions of the intelligent face framing management system to output an optimized centering face framing adjustment to center the face image location in the captured videoframe; and
the processor to execute an AV processing instruction module configured to crop the videoframes to center the face image location in the captured videoframe according to the optimized normalizing face framing adjustment.

5. The information handling system of claim 4 further comprising:
the processor to detect user movement in additional captured videoframes;
the processor to execute the intelligent face framing management system to determine movement of the user to a first side of the captured videoframe that is off-center beyond a threshold amount of shift from center; and
the processor to execute the AV processing instruction module configured to crop the captured videoframes to re-center the face image location in the additional captured videoframes according to a second optimized centering face framing adjustment.

6. The information handling system of claim 1 further comprising:
the processor to execute code instructions of an intelligent face framing management system user interface to receive a user image normalizing setting from a user to set a desired size of the user's image to determine the size percentage range threshold of the face image in the captured videoframe.

7. The information handling system of claim 1 further comprising:
a proximity sensor to detect persons or objects in front of the video camera;
the processor to execute code instructions of an unusual movement detection (UMD) software module to receive proximity sensor distance data and to determine movement of a user location in front of the video camera relative to a face image location within a series of captured videoframes.

8. The information handling system of claim 6 , wherein the proximity sensor is a time-of-flight (TOF) sensor.

9. A method for intelligently face framing a user's image within captured videoframes in a collaboration videoconference session for an information handling system, comprising:
executing code instructions, via a processor, of a multimedia multi-user collaboration application, via a processor, to join a videoconference session of a remotely located computing device;
capturing a videoframe of a user's image via a video camera;
inputting videoframe data, including the user's image, and audiovisual (AV) processing instructions to be applied to videoframes into a trained neural network of the intelligent face framing management system to determine a face center;
executing code instructions of the intelligent face framing management system to output an optimized centering face framing adjustment to center a face image location in the captured videoframe;
executing an AV processing instruction module configured to crop the videoframes to center the face image location in the captured videoframe according to the optimized centering face framing adjustment; and
transmitting, via a network interface device, a processed, encoded media sample, including the captured videoframe to a remotely located computing device participating in the videoconference session.

10. The method of claim 9 further comprising:
detecting user movement, via a proximity sensor, during additional captured videoframes;
the processor to detect user movement with a proximity sensor, during additional captured videoframes;
executing the intelligent face framing management system to output an adjusted optimized centering face framing adjustment in response to the detected user movement.

11. The method of claim 9 further comprising:
detecting user movement, via a proximity sensor, in additional captured videoframes;
executing the intelligent face framing management system to determine movement of the user to a first side of the captured videoframe that is off-center beyond a threshold amount of shift from center; and
executing the AV processing instruction module configured to crop the captured videoframes to re-center the face center location in the captured videoframe according to a second optimized centering face framing adjustment.

12. The method of claim 9 further comprising:
executing code instructions of the trained neural network of the intelligent face framing management system to determine a face image in the user's image of the captured videoframe and to output optimized face framing adjustments to adjust the size of the face image relative to the captured videoframe to within a size percentage range threshold of the face image in the captured videoframe; and
executing an AV processing instruction module configured to execute a digital zoom process on the face image in the captured videoframe to meet the size percentage range threshold pursuant to an optimized normalizing face framing adjustment.

13. The method of claim 12 further comprising:
detecting user movement, via a proximity sensor, in additional captured videoframes;
inputting the captured videoframe data, including the user's image, i

Assignees

• Dell Products Lp

Inventors

• Swierk Todd E
• Viswanathan Iyer Vivek

Classifications

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• H04N23/69

  Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming · CPC title

• H04N23/60

  Control of cameras or camera modules · CPC title

• H04L12/1827

  Network arrangements for conference optimisation or adaptation · CPC title

• H04N7/15

  Conference systems · CPC title