System and method for intelligent appearance monitoring management system for videoconferencing applications

Patent metadata

Field	Value
Publication number	US-11778142-B2
Application number	US-202217828022-A
Country	US
Kind code	B2
Filing date	May 30, 2022
Priority date	Jan 26, 2021
Publication date	Oct 3, 2023
Grant date	Oct 3, 2023

Abstract

A system or method executing an intelligent appearance monitoring management system comprising a processor to execute code instructions of a multimedia multi-user collaboration application to join a videoconference session, a display screen, a speaker, a video camera, and a microphone where the video camera captures a videoframe of a user and the processor to input videoframe data, including the detected user's image, into a trained neural network of the intelligent appearance monitoring management system to generate optimized appearance filtering adjustments indicating detection of a user appearance anomaly in the user's image or altering a user's image in the captured videoframes in response to the user appearance anomaly and prepare those videoframes for transmission.

First claim

What is claimed is:

1. An information handling system executing an intelligent appearance monitoring management system comprising:
a processor executing code instructions of a multimedia multi-user collaboration application to join a videoconference session with a remotely located computing device;
a display screen, a speaker, a video camera, and a microphone;
the video camera configured to capture a videoframe of a user;
the processor executing code instructions to input videoframe data, including a detected user's image and media capture settings into a trained neural network of the intelligent appearance monitoring management system;
the processor executing code instructions of a trained neural network of the intelligent appearance monitoring management system to output detection of a user appearance anomaly in the user's image relative to reference images of the user during the videoconference session;
the video camera configured to capture additional plural videoframes of the videoconference session;
the processor processing the captured plural videoframes and with an indication of the detected user appearance anomaly in the captured additional plural videoframes of the videoconference session; and
a network interface device configured to transmit the processed, captured plural videoframes to the remotely located computing device participating in the videoconference session.

2. The information handling system of claim 1 further comprising:
the processor executing code instructions for the trained neural network to output an anomaly correction adjustment to the user's image in response to the identified user appearance anomaly for the input videoframe data with the indication of the detected user appearance anomaly; and
the processor applying the anomaly correction adjustment to the processed, captured plural additional videoframes.

3. The information handling system of claim 2 , wherein the anomaly correction adjustment utilizes a convolutional neural network to smooth the detected user appearance anomaly in the user's image in the captured plural additional videoframes of the videoconference session.

4. The information handling system of claim 1 , wherein the indication of the detected user appearance anomaly includes identification of a blemish or mark on the user's face detected in the captured plural additional videoframes.

5. The information handling system of claim 1 , wherein the indication of the detected user appearance anomaly includes identification of an anomaly in the user's hair appearance.

6. The information handling system of claim 1 further comprising:
the processor displaying a notification to the user identifying the detected user appearance anomaly for the processed, captured plural additional videoframes having the indication of the detected user appearance anomaly.

7. The information handling system of claim 1 further comprising:
the processor displaying a stock image of the user as the processed, captured plural additional videoframes having the indication of the detected user appearance anomaly to replace the captured videoframes having the detected user appearance anomaly where the user appearance anomaly includes identification of a sneeze or cough by the user in the user's image.

8. The information handling system of claim 1 further comprising:
the processor executing code instructions to input later videoframe data, including the detected user's image and media capture settings into the trained neural network of the intelligent appearance monitoring management system; and
the processor executing code instructions of the trained neural network of the intelligent appearance monitoring management system to output detection that the user appearance anomaly in the user's image relative has been eliminated during the videoconference session and cease adding the indication of the detected user appearance anomaly in the captured additional plural videoframes.

9. A method for intelligently managing a user's appearance in a collaboration videoconference session for an information handling system, comprising:
executing code instructions of a multimedia multi-user collaboration application, via a processor, to join a videoconference session of a remotely located computing device;
capturing a videoframe of a user via a video camera, wherein the videoframe includes the user's image;
inputting videoframe data including the user's image and media capture settings for the captured videoframe into a trained intelligent appearance monitoring management system neural network;
executing code instructions of the trained intelligent appearance monitoring management system neural network, via the processor, to output an optimized appearance filtering adjustment in response to detection of a user appearance anomaly in the user's image relative to reference images of the user during the videoconference session;
capturing plural videoframes, via the video camera, for the videoconference session with the optimized appearance filtering adjustment in response to detection of the user appearance anomaly; and
transmitting, via a network interface device, a processed, encoded media sample, including the captured plural videoframes with the optimized appearance filtering adjustment for the detected user appearance anomaly to the remotely located computing device participating in the videoconference session.

10. The method of claim 9 , wherein the optimized appearance filtering adjustment includes generating a notification to be displayed on the display screen to the user identifying a type of the user appearance anomaly.

11. The method of claim 9 further comprising:
upon detecting user appearance anomaly, replacing the captured plural videoframes in the encoded media sample with a stock user image.

12. The method of claim 9 , wherein the optimized appearance filtering adjustment includes adjusting the user's image to be displayed during the video conference session.

13. The method of claim 12 further comprising:
executing code instructions of an appearance filter module to alter the user's image in the captured plural videoframes to blend out the detected user appearance anomaly in the captured plural videoframes with nearby pixels in the user's image around the detected user appearance anomaly.

14. The method of claim 9 further comprising:
executing code instructions of a boundary detection module to detect a user's image boundary of the user's image within the captured videoframe; and
determining the detected user appearance anomaly relative to the user's image boundary.

15. An information handling system executing an intelligent appearance monitoring management system comprising:
a processor to execute code instructions of a multimedia multi-user collaboration application to join a videoconference session with a remotely located computing device;
a display screen, a speaker, a video camera, and a microphone;
the video camera configured to capture a videoframe of a user;
the processor to execute code instructions to input videoframe data, including a detected user's image and media capture settings into a trained neural network of the intelligent appearance monitoring management system;
the processor to execute code instructions of a trained neural network of the intelligent appearance monitoring management system to output optimized appearance filtering adjustments indicating detection of a user appearance anomaly in the user's image relative to reference images of the user during the videoconference session;
the video camera configured to capture plural, additional videof

Assignees

• Dell Products Lp

Inventors

• Swierk Todd E
• Viswanathan Iyer Vivek

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• H04N7/15 Primary

  Conference systems · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/08

  Learning methods · CPC title