Determining Relevant Feedback Based on Alignment of Feedback with Performance Objectives

Patent metadata

Field	Value
Publication number	US-2016170968-A1
Application number	US-201414566808-A
Country	US
Kind code	A1
Filing date	Dec 11, 2014
Priority date	Dec 11, 2014
Publication date	Jun 16, 2016
Grant date	—

Abstract

Mechanisms, in a natural language processing (NLP) system are provided. The NLP system receives performance data for a performance to be provided by a human performer. The performance data comprises an objective to be achieved by the performance. The NLP system monitors channels of communication to identify natural language statements exchanged over the channels directed to the performance while the performance is being presented. The NLP system extracts feedback information from the natural language statements and generates aggregate feedback information. The NLP system evaluates an alignment of the aggregate feedback information with the objective and outputs a guidance output based on results of the evaluation. The guidance output guides the performer to modify presentation of the performance to more likely achieve the at least one objective based on the aggregate feedback information.

First claim

What is claimed is:

1 . A method, in a natural language processing (NLP) system comprising a processor and a memory, the method comprising:
receiving, by the NLP system, performance data for a performance to be provided by a human performer, wherein the performance data comprises at least one objective to be achieved by the performance; monitoring, by the NLP system, one or more channels of communication to identify natural language statements exchanged over the one or more channels of communication directed to the performance while the performance is being presented; extracting, by the NLP system, feedback information from the natural language statements based on natural language processing of the natural language statements; generating, by the NLP system, aggregate feedback information based on the identified feedback information; evaluating, by the NLP system, an alignment of the aggregate feedback information with the at least one objective in the performance data; and outputting, by the NLP system, a guidance output based on results of evaluating the alignment of the aggregate feedback information with the at least one objective in the performance data, wherein the guidance output guides the performer to modify presentation of the performance to more likely achieve the at least one objective based on the aggregate feedback information.

2 . The method of claim 1 , wherein the at least one object comprises an identifier of at least one type of feedback that the performer of the performance desires to elicit from an audience of the performance.

3 . The method of claim 1 , further comprising:
generating, by the NLP system, an aggregate audience model based on user models of audience members, wherein the aggregate audience model specifies at least one aggregate audience personality trait generated by aggregating personality traits determined from the user models of the audience members.

4 . The method of claim 1 , wherein evaluating the alignment of the aggregate feedback information with the at least one objective in the performance data comprises:
determining first objectives in the at least one objective have been met by the presentation of the performance, based on the aggregate feedback information; determining second objectives in the at least one objective have not been met by the presentation of the performance, based on the aggregate feedback information; associating aggregate personality traits from the aggregate audience model with the aggregate feedback information; and determining reasons for the aggregate feedback information indicating the second objectives have not been met.

5 . The method of claim 4 , wherein the guidance output specifies a degree of credibility to be given by the performer to portions of the aggregate feedback information that indicate the second objectives have not been met.

6 . The method of claim 1 , wherein monitoring one or more channels of communication to identify natural language statements exchanged over the one or more channels of communication directed to the performance while the performance is being presented, comprises:
processing the natural language statements to identify, for each natural language statement, an indicator in the natural language statement that the natural language statement is directed to the performance.

7 . The method of claim 6 , wherein the indicator is one of a hashtag, one or more key terms or phrases that correspond to a topic of the performance, a time stamp corresponding to a time frame in which the performance is presented, or a geolocation that corresponds to a location of the performance.

8 . The method of claim 1 , wherein extracting, by the NLP system, feedback information from the natural language statements based on natural language processing of the natural language statements comprises identifying terms or phrases in the natural language statements indicative of feedback on a performance in either a positive or a negative context.

9 . The method of claim 8 , wherein generating, by the NLP system, aggregate feedback information based on the identified feedback information comprises updating the aggregate feedback information based on the identified terms or phrases, wherein updating the aggregate feedback information comprises at least one of generating a new entry in the aggregate feedback information corresponding to a type of feedback associated with the identified terms or phrases, or updating a count of a number of sources of feedback providing the type of feedback.

10 . The method of claim 9 , wherein outputting, by the NLP system, a guidance output based on the aggregate feedback information, comprises generating the guidance output in response to a number of sources of the feedback reaching or exceeding a predetermined threshold value.

11 . The method of claim 1 , wherein outputting, by the NLP system, a guidance output based on the aggregate feedback information comprises outputting a message or cue instructing a performer of the performance how to modify the performance to correct negative feedback identified in the natural language statements.

12 . The method of claim 1 , wherein the one or more channels of communication comprise at least one instant messaging service through which audience members of an audience of the performance submit instant messages during the performance.

13 . A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device implementing a natural language processing (NLP) system, causes the computing device to:
receive, by the NLP system, performance data for a performance to be provided by a human performer, wherein the performance data comprises at least one objective to be achieved by the performance; monitor, by the NLP system, one or more channels of communication to identify natural language statements exchanged over the one or more channels of communication directed to the performance while the performance is being presented; extract, by the NLP system, feedback information from the natural language statements based on natural language processing of the natural language statements; generate, by the NLP system, aggregate feedback information based on the identified feedback information; evaluate, by the NLP system, an alignment of the aggregate feedback information with the at least one objective in the performance data; and output, by the NLP system, a guidance output based on results of evaluating the alignment of the aggregate feedback information with the at least one objective in the performance data, wherein the guidance output guides the performer to modify presentation of the performance to more likely achieve the at least one objective based on the aggregate feedback information.

14 . The computer program product of claim 13 , wherein the at least one object comprises an identifier of at least one type of feedback that the performer of the performance desires to elicit from an audience of the performance.

15 . The computer program product of claim 13 , wherein the computer readable program further causes the computing device to:
generate, by the NLP system, an aggregate audience model based on user models of audience members, wherein the aggregate audience model specifies at least one aggregate audience personality trait generated by aggregating personality traits determined from the user models of the audience members.

16 . The computer program product of claim 13 , wherein the computer

Assignees

• IBM

Inventors

• Allen Corville O
• Rodriguez Laura J

Classifications

• G06Q10/40

  Business processes related to social networking or social networking services · CPC title

• G06Q10/10

  Office automation; Time management · CPC title

• G09B5/00 Primary

  Electrically-operated educational appliances (working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F) · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G06F40/279 Primary

  Recognition of textual entities · CPC title