Sentiment analysis in a video conference

Patent metadata

Field	Value
Publication number	US-10878226-B2
Application number	US-201715453328-A
Country	US
Kind code	B2
Filing date	Mar 8, 2017
Priority date	Aug 8, 2014
Publication date	Dec 29, 2020
Grant date	Dec 29, 2020

Abstract

In an approach, a computer determines based, at least in part, on a video of an attendee of a video conference, a first sentiment of the attendee wherein the first sentiment includes at least a sentiment from a sentiment analysis of one or more facial expressions of the attendee and a sentiment from a sentiment analysis of a plurality of the attendee's spoken words. The approach includes a computer receiving an indication of an attendee activity in at least a first application in computing devices accessed by the attendee and determining whether the first sentiment of the attendee is related to the video conference based, in part, on the attendee activity in at least the first application. Responsive to determining that the first sentiment of the attendee is not related to the video conference, the computer discards the first sentiment that is unrelated to the video conference.

First claim

What is claimed is:

1. A method for determining a sentiment of an attendee of a video conference, the method comprising:
receiving, by one or more computing devices, a selection by an attendee to display one of: a sentiment of the attendee or a video of the attendee and the sentiment of the attendee to a video conference;
determining, by one or more computing devices, based, at least in part, on a video of an attendee of a video conference, a first sentiment of the attendee, wherein the first sentiment includes a second sentiment of the attendee determined from a sentiment analysis of one or more facial expressions of the attendee and a third sentiment of the attendee determined from a sentiment analysis of a plurality of the attendee's spoken words;
receiving, by one or more computing devices, an indication of an attendee activity in at least a first application in one or more computing devices accessed by the attendee;
determining, by one or more computing devices, whether the first sentiment of the attendee is related to the video conference based, in part, on the attendee activity in at least the first application; and
displaying, by one or more computing devices, the selection by the attendee of the sentiment of the attendee or the video of the attendee and the sentiment of the attendee to the video conference, wherein the displayed sentiment of the attendee includes displaying the first sentiment of the attendee, the second sentiment determined from the sentiment analysis of the one or more facial expressions of the attendee, and the third sentiment of the attendee determined from the sentiment analysis of the plurality of the attendee's spoken words.

2. The method of claim 1 , wherein receiving the indication of the attendee activity in at least the first application includes at least one of:
receiving one or more of attendee input to at least the first application and a displayed output from at least the first application during the video conference or determining at least one of a change in the attendee's eye movement, a change in the attendee's facial expression, and a change in the attendee's vocal intonation is associated with the at least the first application.

3. The method of claim 1 , wherein determining, by one or more computing devices, based, at least in part, on the video of the attendee, the first sentiment of the attendee further comprises using at least one of a facial recognition technique, a natural language processing technique, and a sentiment analysis technique.

4. The method of claim 1 , wherein determining, by one or more computing devices, based, at least in part, on the video of the attendee, a first sentiment of the attendee, further comprises:
compiling, by one or more computing devices, a first sentiment of a plurality of attendees of the video conference to determine a first aggregate sentiment of the plurality of attendees for the video conference, and
displaying, by one or more computing devices, the first aggregated sentiment in the video conference.

5. The method of claim 1 , wherein determining, by one or more computing devices, the first sentiment of the attendee further comprises:
retrieving, by one or more computing devices, historical sentiment data of the attendee and one or more characteristic facial expressions of the attendee; and
determining, by one or more computing devices, the first sentiment of the attendee based, at least in part, on the historical sentiment data of the attendee and the one or more characteristic facial expressions of the attendee.

6. The method of claim 1 , wherein determining whether the first sentiment of the attendee is related to the video conference based, in part, on the attendee activity in the at least the first application further comprises:
determining, by one or more computing devices, in real-time, whether one or more of a plurality of facial expressions of the attendee, one or more eye movements of the attendee, the plurality of the attendee's spoken words, one or more of user input on one or more computing devices by the attendee, viewing a display of a message or viewing a display of an output of an activity from an application on a user interface by the attendee during the video conference is related to one of a real-time aggregate sentiment of the first sentiment a plurality of attendees in the video conference or to a real-time video conference discussion to determine relevance of the first sentiment to the video conference; and
responsive to determining that the one or more attendee activity is relevant to one of the real-time aggregate sentiment or to the real-time discussion in the video conference, determining, by one or more computer processors, the first sentiment of the attendee is related to the video conference.

7. The method of claim 1 , wherein determining based, at least in part, on the video of the attendee of the video conference, the first sentiment of the attendee includes selecting the first sentiment of the attendee that is between the second sentiment of the attendee determined from the sentiment analysis of the one or more facial expressions of the attendee and the third sentiment of the attendee determined from the sentiment analysis of the plurality of the attendee's spoken words.

8. The method of claim 1 , wherein displaying the selection by the attendee of the sentiment of the attendee or the video of the attendee and the sentiment of the attendee to the video conference further comprises:
displaying, by one or more computing devices, the first sentiment as the sentiment of the attendee as a color coded bar, wherein a middle of the bar indicates a number sentiments exhibited by the attendee.

9. The method of claim 1 , wherein determining whether the first sentiment of the attendee is related to the video conference based, in part, on the attendee activity in at least the first application further comprises:
determining, by one or more computing devices, that the first sentiment of the attendee is not related to the video conference; and
determining, by one or more computing devices, not to display the first sentiment of the attendee, the second sentiment of the attendee, and the third sentiment of the attendee when the attendee activity in at least the first application by the attendee is not related to the video conference.

10. The method of claim 1 , wherein determining based, at least in part, on the video of the attendee of the video conference, the first sentiment of the attendee, wherein the first sentiment includes at least the second sentiment of the attendee determined from the sentiment analysis of the one or more facial expressions of the attendee and the third sentiment of the attendee determined from the sentiment analysis of the plurality of the attendee's spoken words further comprises:
determining, by one or more computing devices, that the second sentiment of the attendee determined from the sentiment analysis of the one or more facial expressions of the attendee significantly differs from the third sentiment of the attendee determined from the sentiment analysis of the plurality of the attendee's spoken words; and
displaying, by one or more computing devices, both the second sentiment and the third sentiment in yellow, wherein the yellow color indicates a conflict in the second sentiment of the attendee determined from the sentiment analysis of the one or more facial expressions of the attendee and the third sentiment of the attendee determined from the sentiment analysis of the plurality of the attendee's spoken words.

11. A computer program product for determining a sentiment of an attendee of a video conference, the computer program product comprising:
o

Assignees

• IBM

Inventors

• Cunico Hernan A
• Silva Asima

Classifications

• H04N7/147 Primary

  Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q) · CPC title

• G06V40/176 Primary

  Dynamic expression · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title

• G10L15/1815

  Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning · CPC title

• H04L65/1066

  Session management · CPC title