Automatic image generator using meeting content

Patent metadata

Field	Value
Publication number	US-11937015-B2
Application number	US-202217884669-A
Country	US
Kind code	B2
Filing date	Aug 10, 2022
Priority date	Aug 10, 2022
Publication date	Mar 19, 2024
Grant date	Mar 19, 2024

Abstract

Methods and systems disclosed herein describe generating virtual backgrounds for video communications. A virtual background generator may monitor a user's calendar and/or inbox for meetings. The virtual background generator may analyze the context of calendar invites and/or scheduled meetings to generate one or more virtual backgrounds for a video conference. A first background may be displayed for the video conference. Additionally, the virtual background generator may update the virtual background based on an analysis of one or more topics being discussed during the video conference. Based on the analysis of the one or more topics being discussed, the virtual background generator may generate a second virtual background to replace the first virtual background.

First claim

What is claimed is:

1. A computer-implemented method comprising:
analyzing, by a computing device and using a machine learning model, one or more criteria of a meeting, wherein the machine learning model is configured to identify confidential information in the one or more criteria;
generating, by the computing device and using the machine learning model, one or more first backgrounds for a video conference based on the one or more criteria contained in the meeting, wherein the machine learning model is configured to exclude the confidential information when generating the one or more first backgrounds;
causing, by the computing device, display of a first background of the one or more first backgrounds during the video conference;
receiving, by the computing device and during the video conference, one or more streams of data from one or more client devices attending the video conference;
analyzing, by the computing device, the one or more streams of data to identify one or more topics being discussed during the video conference;
generating, by the computing device and using the machine learning model, one or more second backgrounds for the video conference based on the one or more topics being discussed during the video conference, wherein the one or more second backgrounds update the first background currently being displayed during the video conference; and
causing, by the computing device, display of a second background, of the one or more second backgrounds, during the video conference.

2. The computer-implemented method of claim 1 , wherein the meeting is obtained by:
accessing, by the computing device and via an application programming interface (API), a user's calendar.

3. The computer-implemented method of claim 1 , wherein the one or more criteria comprises at least one of:
an attachment to the meeting;
an agenda associated with the meeting;
a list of attendees for the meeting;
a message body;
a set of e-mail addresses associated with the meeting;
a subject message associated with the meeting;
a word spoken by an attendee during the meeting;
a phrase spoken by an attendee during the meeting; or
a gesture made by an attendee during the meeting.

4. The computer-implemented method of claim 1 , wherein the video conference comprises at least one of:
a webinar; or
an online meeting.

5. The computer-implemented method of claim 1 , wherein the one or more first backgrounds comprise one or more of:
a static image;
a dynamic image;
an animated image;
a video;
a graphics interchange format (GIF) image;
a meeting agenda;
an action item list for the meeting;
a presentation associated with the meeting; or
an overlay for an existing background.

6. The computer-implemented method of claim 1 , wherein the causing display of the first background during the video conference comprises:
sending, by the computing device and to a client device of the one or more client devices, a request for a selection of the one or more first backgrounds; and
receiving, by the computing device and from the client device, a selection of the first background, wherein the first background is displayed based on the selection.

7. The computer-implemented method of claim 1 , wherein the one or more streams of data comprise at least one of:
an audio stream;
a video stream; or
a text stream.

8. The computer-implemented method of claim 7 , wherein the analyzing the one or more streams of data to identify one or more topics being discussed during the video conference comprises at least one of:
transcribing, using a speech-to-text algorithm, the audio stream;
inspecting, using an image analysis process, the video stream to identify one or more gestures in the video stream; or
analyzing, using natural language processing, the text stream.

9. The computer-implemented method of claim 1 , wherein the one or more streams of data comprise messages sent in a chat by one or more participants of the meeting.

10. A non-transitory computer-readable media storing instructions that, when executed, cause a computing device to:
analyze, by a computing device and using a machine learning model, one or more criteria of a meeting, wherein the machine learning model is configured to identify confidential information in the one or more criteria;
generate, by the computing device and using the machine learning model, one or more first backgrounds for a video conference based on the one or more criteria contained in the meeting, wherein the machine learning model is configured to exclude the confidential information when generating the one or more first backgrounds;
cause, by the computing device, display of a first background of the one or more first backgrounds during the video conference;
receive, during the video conference, one or more streams of data from one or more devices attending the video conference;
analyze the one or more streams of data to identify one or more topics being discussed during the video conference;
parse, using natural language processing, the one or more topics to generate a set of word embeddings;
input the set of word embeddings to the machine learning model;
generate, using the machine learning model, one or more backgrounds for the video conference based on the set of word embeddings, wherein the one or more backgrounds update the first background currently being displayed during the video conference; and
cause display of a second background, of the one or more backgrounds, during the video conference.

11. The non-transitory computer-readable media of claim 10 , wherein the one or more streams of data comprise an audio stream.

12. The non-transitory computer-readable media of claim 11 , wherein the instructions, when executed, cause the computing device to:
transcribe, using a speech-to-text algorithm, the audio stream, wherein the parsing the one or more topics to generate the set of word embeddings further comprises analyzing the transcribed audio stream.

13. The non-transitory computer-readable media of claim 10 , wherein the one or more streams of data comprise a video stream.

14. The non-transitory computer-readable media of claim 13 , wherein the instructions, when executed, cause the computing device to:
analyze, using an image analysis process, the video stream; and
identify one or more gestures in the video stream.

15. The non-transitory computer-readable media of claim 14 , wherein the one or more gestures causes at least one of:
a first object in the first background to be displayed in a different location in the second background;
a new slide, of a presentation, to be displayed as the second background; or
an agenda item to be updated in the second background.

16. A computing device comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the computing device to:
analyze, by a computing device and using a machine learning model, one or more criteria of a meeting, wherein the machine learning model is configured to identify confidential information in the one or more criteria;
generate, by the computing device and using the machine learning model, one or more first backgrounds for a video conference based on the one or more criteria contained in the meeting, wherein the machine learning model is configured to exclude the confidential information when generating the one or more first backgrounds;
cause, by the computing device, display of a first background of the one or more first backgrounds during the video conference;
receive, during the video conferenc

Assignees

• Capital One Services Llc

Inventors

• Kare Sarvani
• Tran Daniel
• Adcock Lee
• Shyamala Geeta
• Polk Benjamin

Classifications

• H04N7/15 Primary

  Conference systems · CPC title

• G06V10/70

  using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) · CPC title

• H04L12/1822

  Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission · CPC title

• H04N21/4394

  involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58) · CPC title

• H04N21/44008

  involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title