Informing a support agent of a paralinguistic emotion signature of a user

Patent metadata

Field	Value
Publication number	US-10192569-B1
Application number	US-201615336113-A
Country	US
Kind code	B1
Filing date	Oct 27, 2016
Priority date	Oct 27, 2016
Publication date	Jan 29, 2019
Grant date	Jan 29, 2019

Abstract

Techniques are disclosed for assisting a support agent based on paralinguistic information determined from a user interacting with a software application. One embodiment presented herein includes a computer-implemented method. The method includes receiving, at a computing device, an audio stream including audio of a user interacting with an application. The method also includes evaluating the audio stream to identify a collection of paralinguistic information present in the audio stream. The paralinguistic information includes a set of descriptors characterizing acoustic aspects of the audio that are distinct from verbal content of the audio. The method further includes determining, from the paralinguistic information, attribute measures associated with the user interacting with the application. Upon receiving a request to initiate a support encounter, the method includes informing a support agent assigned to handle the support encounter of the attribute measures determined from the paralinguistic information.

First claim

What is claimed is:

1. A computer-implemented method for assisting a support agent assigned to interact with a user during a support encounter, comprising:
receiving, at a computing device, an audio stream comprising audio of a user interacting with an application;
evaluating the audio stream to identify a collection of paralinguistic information present in the audio stream, wherein the paralinguistic information comprises a set of descriptors characterizing acoustic aspects of the audio that are distinct from verbal content of the audio;
determining, from the paralinguistic information, one or more attribute measures associated with the user interacting with the application; and
upon receiving a request to initiate a support encounter:
generating, before a support agent assigned to handle the support encounter interacts with the user and based on evaluating at least one of the attribute measures with an uplift model, a set of activities for the support agent to use when interacting with the user that increases a likelihood of achieving a specified outcome for the support encounter;
providing information content on a support agent interface before the support agent interacts with the user, the information content comprising the one or more attribute measures determined from the paralinguistic information and the generated set of activities, wherein the one or more attribute measures comprises an emotional state of the user responsive to at least one content item provided by the application; and
generating, before the support agent interacts with the user, a mockup of the application on the support agent interface, the mockup comprising the at least one content item.

2. The computer-implemented method of claim 1 , wherein the one or more attribute measures further comprises at least one of a gender of the user and an age of the user.

3. The computer-implemented method of claim 1 , wherein the emotional state of the user is a negative emotional state of the user in response to the at least one content item provided by the application.

4. The computer-implemented method of claim 1 , wherein the set of activities comprises at least one of identifying first content items to the user, and avoiding presenting second content items to the user.

5. The computer-implemented method of claim 1 , further comprising:
determining, based on the one or more attribute measures, at least one topic associated with the user's request to initiate the support encounter, wherein the at least one topic is not identified in the verbal content of the audio; and
notifying the support agent of the at least one topic before the support agent interacts with the user.

6. The computer-implemented method of claim 5 , wherein the at least one topic is determined without prompting the user to explain a reason for initiating the request for the support encounter.

7. The computer-implemented method of claim 1 , wherein the emotional state of the user is a positive emotional state of the user in response to the at least one content item provided by the application.

8. A computing device for assisting a support agent assigned to interact with a user during a support encounter, the computing device comprising:
a memory; and
a processor configured to:
receive an audio stream comprising audio of a user interacting with an application;
evaluate the audio stream to identify a collection of paralinguistic information present in the audio stream, wherein the paralinguistic information comprises a set of descriptors characterizing acoustic aspects of the audio that are distinct from verbal content of the audio;
determine, from the paralinguistic information, one or more attribute measures associated with the user interacting with the application; and
upon receiving a request to initiate a support encounter:
generate, before a support agent assigned to handle the support encounter interacts with the user and based on evaluating at least one of the attribute measures with an uplift model, a set of activities for the support agent to use when interacting with the user that increases a likelihood of achieving a specified outcome for the support encounter;

provide information content on a support agent interface before the support agent interacts with the user, the information content comprising the one or more attribute measures determined from the paralinguistic information and the generated set of activities, wherein the one or more attribute measures comprises an emotional state of the user responsive to at least one content item provided by the application, and
generate, before the support agent interacts with the user, a mockup of the application on the support agent interface, the mockup comprising the at least one content item.

9. The computing device of claim 8 , wherein the one or more attribute measures further comprises at least one of a gender of the user and an age of the user.

10. The computing device of claim 8 , wherein the emotional state of the user is a negative emotional state of the user in response to the at least one content item provided by the application.

11. The computing device of claim 8 , wherein the set of activities comprises at least one of identifying first content items to the user, and avoiding presenting second content items to the user.

12. The computing device of claim 8 , wherein the processor is further configured to:
determine, based on the one or more attribute measures, at least one topic associated with the user's request to initiate the support encounter, wherein the at least one topic is not identified in the verbal content of the audio; and
notify the support agent of the at least one topic before the support agent interacts with the user.

13. The computing device of claim 12 , wherein the at least one topic is determined without prompting the user to explain a reason for initiating the request for the support encounter.

14. The computing device of claim 8 , wherein the emotional state of the user is a positive emotional state of the user in response to the at least one content item provided by the application.

15. A non-transitory computer-readable medium comprising instructions that when executed by a computing device cause the computing device to perform a method for assisting a support agent assigned to interact with a user during a support encounter, the method comprising:
receiving an audio stream comprising audio of a user interacting with an application;
evaluating the audio stream to identify a collection of paralinguistic information present in the audio stream, wherein the paralinguistic information comprises a set of descriptors characterizing acoustic aspects of the audio that are distinct from verbal content of the audio;
determining, from the paralinguistic information, one or more attribute measures associated with the user interacting with the application; and
upon receiving a request to initiate a support encounter:
generating, before a support agent assigned to handle the support encounter interacts with the user and based on evaluating at least one of the attribute measures with an uplift model, a set of activities for the support agent to use when interacting with the user that increases a likelihood of achieving a specified outcome for the support encounter;
providing information content on a support agent interface before the support agent interacts with the user, the information content comprising the one or more attribute measures determined from the paralinguistic information and the generated set of activities, wherein the one or more attribute measures comprises an emot

Assignees

• Intuit Inc

Inventors

• Indyk Benjamin
• Podgorny Igor A
• Chan Raymond

Classifications

• G10L25/63

  for estimating an emotional state · CPC title

• H04M3/42042 Primary

  Notifying the called party of information on the calling party (details within substation equipment H04M1/57, signalling details H04Q3/72) · CPC title

• G10L25/87 Primary

  Detection of discrete points within a voice signal · CPC title

• G10L15/1815

  Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning · CPC title

• H04M3/5183

  Call or contact centers with computer-telephony arrangements · CPC title