Virtual meeting coaching with content-based evaluation

Patent metadata

Field	Value
Publication number	US-12374232-B2
Application number	US-202318104122-A
Country	US
Kind code	B2
Filing date	Jan 31, 2023
Priority date	Nov 7, 2022
Publication date	Jul 29, 2025
Grant date	Jul 29, 2025

Abstract

Methods and systems provide for virtual meeting coaching with content-based evaluation. In one embodiment, the system receives a set of coaching items including a number of questions each associated with an expected answer; connects to a coaching session including one or more participants and a virtual coaching agent; for each question and for at least a subset of the participants: transmitting the question, by the virtual coaching agent, to the client device used by the participant; receiving an answer to the question by the participant, the answer including media of the participant; receiving text of utterances spoken by the participant during the answer; generating one or more evaluation scores for the answer based on evaluating at least the content of the answer to the question; and transmitting an overall evaluation score for each of the subset of participants based on the generated evaluation scores for that participant.

First claim

What is claimed is:

1. A method, comprising:
receiving a set of coaching items comprising a plurality of questions each associated with an expected answer;
connecting to a coaching session comprising a participant using a client device and a virtual coaching agent;
for each of one or more questions from the plurality of questions:
transmitting, for output by the client device, the question to the client device, the question being transmitted as uttered by the virtual coaching agent;
receiving, from the client device, an answer to the question by the participant, the answer comprising video output of the participant captured by a camera of the client device;
receiving text of utterances spoken by the participant during the answer, the utterances captured by a microphone of the client device, the utterances converted to text using a processor of the client device; and
generating one or more evaluation scores for the answer to the question based on evaluating the video output and the text of utterances, wherein at least one of the one or more evaluation scores is based on a tonal style of the participant, wherein the tonal style accounts for a language spoken by the participant; and

transmitting, to the client device, an overall evaluation score pertaining to the coaching session for the participant, the overall evaluation score being determined based on the generated evaluation scores for the participant.

2. The method of claim 1 , wherein the set of coaching items is a scenario where the plurality of questions and the plurality of associated expected answers all relate to a common context.

3. The method of claim 1 , wherein for each of the one or more questions from the plurality of questions, generating the one or more evaluation scores is performed in real-time.

4. The method of claim 1 , wherein the virtual coaching agent is represented in visual media by a digital avatar.

5. The method of claim 4 , wherein the digital avatar is triggered based on vocal speech generated for the virtual coaching agent.

6. The method of claim 1 , wherein generating the one or more evaluation scores is further based on evaluating a geographic location of the participant.

7. The method of claim 1 , wherein the answer further comprises video of the participant, and wherein generating the one or more evaluation scores is further based on evaluating a visual expression of the participant from the video of the answer.

8. The method of claim 1 , wherein each expected answer comprises one or more key points, and each key point comprises a headline and one or more conversation sentences.

9. The method of claim 1 , wherein each expected answer comprises one or both of: one or more expected expressions, and one or more expected sentiments.

10. The method of claim 9 , further comprising:
receiving a user interface interaction from one of the participants requesting display of a headline associated with each of one or more key points for the expected answer;
determining permission to display the headline for the participant; and
transmitting, to the client device, the headline associated with each of the one or more key points to be displayed at the client device.

11. The method of claim 1 , wherein evaluating the video output and the text of utterances comprises comparing the utterances of the answer to the text of the expected answer to determine a coverage of the answer, wherein at least one of the evaluation scores is generated based on the coverage of the answer.

12. The method of claim 1 , further comprising;
prior to transmitting a next question to the client device, determining that the answer to a question by the participant has terminated.

13. The method of claim 12 , wherein determining that the answer has terminated comprises:
detecting a pause in speech of the participant beyond a specified pause threshold; and
detecting a segmentation boundary mark for a sentence uttered by the participant as part of the answer.

14. The method of claim 1 , further comprising:
receiving, from the client device, a question from one of the participants to the virtual coaching agent;
determining a similarity match of the question from the participant to an expected question from a set of expected questions, each expected question being associated with a predefined answer; and
transmitting, to the client device, the predefined answer associated with the expected question, the predefined answer being transmitted as uttered by the virtual coaching agent.

15. The method of claim 1 , further comprising:
receiving, from the client device, a question from one of the participants to the virtual coaching agent;
determining that there is no similarity match of the question from the participant to any expected questions from a set of expected questions; and
transmitting, to the client device, a canned answer from a set of one or more canned answers to unexpected questions, the canned answer being transmitted as uttered by the virtual coaching agent.

16. A communication system comprising one or more processors configured to perform operations of:
receiving a set of coaching items comprising a plurality of questions each associated with an expected answer;
connecting to a coaching session comprising a participant using a client device and a virtual coaching agent;
for each of one or more questions from the plurality of questions:
transmitting, for output by the client device, the question to the client device, the question being transmitted as uttered by the virtual coaching agent;
receiving, from the client device, an answer to the question by the participant, the answer comprising video output of the participant captured by a camera of the client device;
receiving text of utterances spoken by the participant during the answer, the utterances captured by a microphone of the client device, the utterances converted to text using a processor of the client device; and
generating one or more evaluation scores for the answer to the question based on evaluating the video output and the text of utterances, wherein at least one of the one or more evaluation scores is based on a tonal style of the participant, wherein the tonal style accounts for a language spoken by the participant; and

transmitting, to the client device, an overall evaluation score pertaining to the coaching session for the participant, the overall evaluation score being determined based on the generated evaluation scores for the participant.

17. The communication system of claim 16 , wherein generating the one or more evaluation scores comprises generating an evaluation score for one or more of:
an average number of filler words within a designated window of time, an average talk speed, an average sentence length, a talk-listen ratio, a longest sentence, and an amount of speaker interruptions.

18. A non-transitory computer-readable medium containing instructions, that when executed by a processor, cause the processor to perform operations comprising:
receiving a set of coaching items comprising a plurality of questions each associated with an expected answer;
connecting to a coaching session comprising a participant using a client device and a virtual coaching agent;
for each of one or more questions from the plurality of questions:
transmitting, for output by the client device, the question to the client device, the question being transmitted as uttered by the virtual coaching agent;
receiving, from the client device, an answer to the question by the participant, the answer comprising video output o

Assignees

• Zoom Communications Inc

Inventors

• Ji Wei
• Lee Bilung
• Parthasarathy Vijay
• Rajkumar David Dharmendran
• Yacovone Mark

Classifications

• G10L13/02

  Methods for producing synthetic speech; Speech synthesisers · CPC title

• G10L25/78

  Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10) · CPC title

• G10L15/05

  Word boundary detection · CPC title

• G06V40/176

  Dynamic expression · CPC title

• G10L25/57

  for processing of video signals · CPC title