Computerized intelligent assistant for conferences

Patent metadata

Field	Value
Publication number	US-11688399-B2
Application number	US-202017115293-A
Country	US
Kind code	B2
Filing date	Dec 8, 2020
Priority date	May 4, 2018
Publication date	Jun 27, 2023
Grant date	Jun 27, 2023

Abstract

A method for facilitating a remote conference includes receiving a digital video and a computer-readable audio signal. A face recognition machine is operated to recognize a face of a first conference participant in the digital video, and a speech recognition machine is operated to translate the computer-readable audio signal into a first text. An attribution machine attributes the text to the first conference participant. A second computer-readable audio signal is processed similarly, to obtain a second text attributed to a second conference participant. A transcription machine automatically creates a transcript including the first text attributed to the first conference participant and the second text attributed to the second conference participant.

First claim

The invention claimed is:

1. At least one computer-storage device embodying computer-usable instructions which, when executed by at least one processor, implement a method for providing computer assistance to participants of a conference, the method comprising:
receiving audio of the participants;
identifying the participants;
translating the audio into text;
generating a transcript of the conference;
tracking at least one of an arrival or a departure of a first participant during the conference;
generating a tailored transcript for the first participant based at least on the transcript of the conference and further selectively based on either the arrival or the departure of the first participant; and
providing the tailored transcript to the first participant via at least one of a display, an audio output, an email, a link, a text message, a chat message, an electronic message, or a notification message.

2. The at least one computer-storage device of claim 1 , the method further comprising attributing portions of the text to one or more participants.

3. The at least one computer-storage device of claim 2 , the transcript generated based at least in part on the attribution of portions of the text to the one or more participants.

4. The at least one computer-storage device of claim 1 , the participants comprising local participants and/or remote participants.

5. The at least one computer-storage device of claim 1 , wherein the participants are identified based at least in part on the audio.

6. The at least one computer-storage device of claim 1 , comprising receiving video of the participants, wherein the participants are identified based at least in part on the audio and/or the video.

7. The at least one computer-storage device of claim 1 , wherein generating the tailored transcript comprises determining a time period during the conference when the first participant was absent, based at least in part on the arrival or the departure of the first participant, the tailored transcript comprising:
a filtered portion of the transcript, a summarized portion of the transcript, or a contextualized portion of the transcript; or
the transcript comprising an indication of a portion of the transcript that corresponds to the time period during the conference when the first participant was absent.

8. The at least one computer-storage device of claim 6 , wherein tracking at least one of the arrival or the departure of the first participant comprises determining an availability of the first participant to participate in the conference, the availability determined based on at least one of determining whether the first participant is logged in to the conference, determining whether the first participant is detectable in the audio, determining whether the first participant is detectable in the video, determining a physical location of the first participant, or detecting an audiovisual cue indicating that the first participant left the conference.

9. A computerized conference assistant for providing assistance to participants of a conference, the computerized conference assistant comprising:
at least one processor;
at least one storage device storing computer-usable instructions which, when executed by the at least one processor, implement operations comprising:
receive audio of the participants;
identify the participants;
translate the audio into text;
generate a transcript of the conference;
track at least one of an arrival or a departure of a first participant during the conference;
generate a tailored transcript for the first participant based at least on the transcript of the conference and further selectively based on either the arrival or the departure of the first participant; and
providing the tailored transcript to the first participant via at least one of a display, an audio output, an email, a link, a text message, a chat message, an electronic message, or a notification message.

10. The computerized conference assistant of claim 9 , the operations further comprising attributing portions of the text to one or more participants, the transcript generated based at least in part on the attribution of portions of the text to the one or more participants.

11. The computerized conference assistant of claim 9 , wherein the participants are identified based at least in part on the audio.

12. The computerized conference assistant of claim 9 , the operations further comprising:
receive video of the participants; and
identify the participants based at least in part on the audio and/or the video.

13. The computerized conference assistant of claim 9 , wherein the tailored transcript is modified at least in part based on a time period during the conference when the first participant was absent, the time period based at least in part on the arrival or the departure of the first participant.

14. The computerized conference assistant of claim 13 , the operations further comprising:
determining one or more times in the transcript comprising content that is of interest to the first participant;
wherein the tailored transcript comprises:
a filtered portion of the transcript, a summarized portion of the transcript, or a contextualized portion of the transcript, associated with the determined one or more times in the transcript; or
the transcript and an indication of a portion of the transcript that corresponds to the time period during the conference when the first participant was absent.

15. The computerized conference assistant of claim 12 , wherein track at least one of the arrival or the departure of the first participant comprises determine whether the first participant is logged in to the conference, determine whether the first participant is detectable in the audio, determine whether the first participant is detectable in the video, determine a physical location of the first participant, or detect an audiovisual cue indicating that the first participant left the conference.

16. A method for providing computer assistance to participants of a conference, comprising:
receiving audio of the participants;
identifying the participants;
generating a record of the conference;
attributing portions of the record to one or more participants;
tracking at least one of an arrival or a departure of a first participant during the conference;
generating a tailored record of the conference for the first participant based at least on the record of the conference and further selectively based on either the arrival or the departure of the first participant; and
providing the tailored record of the conference to the first participant via at least one of a display, an audio output, an email, a link, a text message, a chat message, an electronic message, or a notification message.

17. The method of claim 16 , the tailored record of the conference comprising audio, video, and/or text translated from the audio.

18. The method of claim 16 , wherein the participants are identified based at least in part on the audio and/or a received video of the conference.

19. The method of claim 16 , wherein the tailored record of the conference focuses on a time period during the conference when the first participant was absent, based at least in part on the arrival or the departure of the first participant.

20. The method of claim 19 , wherein the tailored record of the conference comprises a portion of the record of the conference, a summarized portion of the record of the conference, or a contextualized portion of the record of the confere

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Diamant Adi
• Master Ben-Dor Karen
• Krupka Eyal
• Halaly Raz
• Smolin Yoni
• Gurvich Ilya
• Hurvitz Aviv
• Qin Lijuan
• Xiong Wei
• Zhang Shixiong
• Wu Lingfeng
• Xiao Xiong
• Leichter Ido
• David Moshe
• Huang Xuedong
• Agarwal Amit Kumar

Classifications

• G10L15/26 Primary

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• G10L17/00

  Speaker identification or verification techniques · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title

• H04N7/15

  Conference systems · CPC title

• H04N7/147 Primary

  Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q) · CPC title