Intelligent performance rating

Patent metadata

Field	Value
Publication number	US-11244166-B2
Application number	US-201916684703-A
Country	US
Kind code	B2
Filing date	Nov 15, 2019
Priority date	Nov 15, 2019
Publication date	Feb 8, 2022
Grant date	Feb 8, 2022

Abstract

Aspects of the invention include receiving performance data comprising video data and audio data associated with a performance by a performer, wherein the video data comprises audience video data for the performance, determining a performer skill score based on a feature vector generated by a performance skill machine learning model, the feature vector comprising a plurality of features extracted from the performance data, parsing the performance data into a plurality of performance segments, analyzing the performance data to determine a performer emotion of the performer for each performance segment, determining an audience emotion based on the audience video data for one or more audience members for each of the performance segments, comparing the audience emotion to the performer emotion for each performance segment to determine an emotion accuracy score, and determining a final score for the performance based on the performer skill score and the emotion accuracy score.

First claim

What is claimed is:

1. A computer-implemented method for performance rating, the method comprising:
receiving performance data comprising video data and audio data associated with a performance by a performer, wherein the video data comprises audience video data for the performance;
determining a performer skill score based on a feature vector generated by a performance skill machine learning model, the feature vector comprising a plurality of features extracted from the performance data;
parsing the performance data into a plurality of performance segments;
analyzing the performance data to determine a performer emotion of the performer for each performance segment in the plurality of performance segments;
determining an audience emotion based on the audience video data for one or more audience members for each of the performance segments in the plurality of performance segments;
comparing the audience emotion to the performer emotion for each performance segment in the plurality of performance segments to determine an emotion accuracy score; and
determining a final score for the performance based at least in part on the performer skill score and the emotion accuracy score.

2. The computer-implemented method of claim 1 , further comprising:
receiving historical performance data associated with a historical performance, the historical performance data comprises historical comment audio data comprising one or more historical comments made by one or more judges associated with the historical performance,
wherein the historical performance is associated with the performance; and
converting the historical comment audio data to textual data;
analyzing the textual data to train a machine learning model; and
generating, by the machine learning model, one or more comments for the performance based on the one or more historical comments for the historical performance.

3. The computer-implemented method of claim 1 , wherein the audience emotion is further determined based on physiological data obtained by one or more sensors associated with each of the one or more audience members.

4. The computer-implemented method of claim 1 , wherein the performance data further comprises online user data; and the method further comprises:
determining one or more user actions taken by a user during viewing the performance; and
determining a user score for the performance based on the one or more user actions.

5. The computer-implemented method of claim 4 , wherein the final score for the performance is further based on the user score for the performance.

6. The computer-implemented method of claim 4 , wherein the one or more user actions comprises one or more of a fast-forwarding, a pausing, and a rewinding of the performance by the user.

7. The computer-implemented method of claim 1 , wherein the plurality of features comprises one or more of timing, pitch, intonation, volume, and timber of the performance.

8. A system comprising:
a memory having computer readable instructions; and
one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:
receiving performance data comprising video data and audio data associated with a performance by a performer, wherein the video data comprises audience video data for the performance;
determining a performer skill score based on a feature vector generated by a performance skill machine learning model, the feature vector comprising a plurality of features extracted from the performance data;
parsing the performance data into a plurality of performance segments;
analyzing the performance data to determine a performer emotion of the performer for each performance segment in the plurality of performance segments;
determining an audience emotion based on the audience video data for one or more audience members for each of the performance segments in the plurality of performance segments;
comparing the audience emotion to the performer emotion for each performance segment in the plurality of performance segments to determine an emotion accuracy score; and
determining a final score for the performance based at least in part on the performer skill score and the emotion accuracy score.

9. The system of claim 8 , further comprising:
receiving historical performance data associated with a historical performance, the historical performance data comprises historical comment audio data comprising one or more historical comments made by one or more judges associated with the historical performance; and
wherein the historical performance is associated with the performance;
converting the historical comment audio data to textual data;
analyzing the textual data to train a machine learning model; and
generating, by the machine learning model, one or more comments for the performance based on the one or more historical comments for the historical performance.

10. The system of claim 8 , wherein the audience emotion is further determined based on physiological data obtained by one or more sensors associated with each of the one or more audience members.

11. The system of claim 8 , wherein the performance data further comprises online user data; and the method further comprises:
determining one or more user actions taken by a user during viewing the performance; and
determining a user score for the performance based on the one or more user actions.

12. The system of claim 11 , wherein the final score for the performance is further based on the user score for the performance.

13. The system of claim 11 , wherein the one or more user actions comprises one or more of a fast-forwarding, a pausing, and a rewinding of the performance by the user.

14. A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations comprising:
receiving performance data comprising video data and audio data associated with a performance by a performer, wherein the video data comprises audience video data for the performance;
determining a performer skill score based on a feature vector generated by a performance skill machine learning model, the feature vector comprising a plurality of features extracted from the performance data;
parsing the performance data into a plurality of performance segments;
analyzing the performance data to determine a performer emotion of the performer for each performance segment in the plurality of performance segments;
determining an audience emotion based on the audience video data for one or more audience members for each of the performance segments in the plurality of performance segments;
comparing the audience emotion to the performer emotion for each performance segment in the plurality of performance segments to determine an emotion accuracy score; and
determining a final score for the performance based at least in part on the performer skill score and the emotion accuracy score.

15. The computer program product of claim 14 , further comprising:
receiving historical performance data associated with a historical performance, the historical performance data comprises historical comment audio data comprising one or more historical comments made by one or more judges associated with the historical performance; and
wherein the historical performance is associated with the performance;
converting the historical comment audio data to textual data;
analyzing the textual data to tr

Assignees

• IBM

Inventors

• Zhang Zhen
• Zheng Xin
• He Hai
• Li ji dong
• Lu Xiaowan

Classifications

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title