Emotionally aware intelligent voice interface

Patent metadata

Field	Value
Publication number	US-12309319-B2
Application number	US-202217870071-A
Country	US
Kind code	B2
Filing date	Jul 21, 2022
Priority date	Jul 22, 2021
Publication date	May 20, 2025
Grant date	May 20, 2025

Abstract

A method for responding to inferred caller states during dialog with an intelligent voice interface configured to lead callers through pathways of an algorithmic dialog may include, during a voice communication with a caller via a caller device, receiving from the caller device caller input data indicative of a voice input of the caller, and determining, by processing the caller input data, an inferred state of the caller. Determining the inferred state of the caller may include analyzing one or more characteristics, other than textual content, of the voice input. The method may also include selecting a pathway through the algorithmic dialog based upon the inferred state of the caller.

First claim

The invention claimed is:

1. A computer-implemented method for responding to inferred caller states during dialog with an intelligent voice interface, wherein the intelligent voice interface is configured to lead callers through pathways of an algorithmic dialog that includes one or more available voice prompts for requesting caller information, the computer-implemented method comprising, during a voice communication with a caller via a caller device:
receiving from the caller device, by one or more processors implementing the intelligent voice interface, caller input data indicative of a voice input of the caller;
determining, by the one or more processors processing the caller input data, an inferred state of the caller, wherein determining the inferred state of the caller includes analyzing one or more characteristics, other than textual content, of the voice input to determine a caller state score based on a plurality of scores corresponding to a plurality of events exhibiting the one or more characteristics of the voice input, and determine the inferred state in response to determining that the caller state score exceeds a threshold score, wherein the plurality of events are related to two or more of: a speed, a pitch, or a volume of a voice of the caller; and
selecting, by the one or more processors, a pathway through the algorithmic dialog based upon the inferred state of the caller.

2. The computer-implemented method of claim 1 , wherein the one or more characteristics include loudness of the voice of the caller.

3. The computer-implemented method of claim 1 , wherein the one or more characteristics include the pitch of the voice of the caller.

4. The computer-implemented method of claim 1 , wherein the one or more characteristics include rapidity with which the caller speaks.

5. The computer-implemented method of claim 1 , wherein determining the inferred state of the caller includes determining that the caller is impatient, angry, or frustrated.

6. The computer-implemented method of claim 1 , wherein determining the inferred state of the caller includes determining that the caller is happy, content, or satisfied.

7. The computer-implemented method of claim 1 , wherein selecting the pathway through the algorithmic dialog includes bypassing one or more voice prompts based upon the inferred state of the caller.

8. The computer-implemented method of claim 1 , wherein selecting the pathway through the algorithmic dialog includes:
generating, by the one or more processors, a voice prompt that asks whether the caller would like to be transferred to a human representative; and
sending, by the one or more processors, the voice prompt to the caller device.

9. The computer-implemented method of claim 1 , wherein determining the inferred state of the caller includes analyzing (i) the one or more characteristics of voice input and (ii) textual content of the voice input.

10. The computer-implemented method of claim 1 , further comprising:
evaluating, by the one or more processors, the voice communication with the caller based upon the inferred state.

11. The computer-implemented method of claim 1 , wherein the caller information includes information associated with a caller account, a caller claim, caller personal information, an order being placed by the caller, and/or an event involving the caller.

12. The computer-implemented method of claim 1 , wherein:
receiving the caller input data indicative of the voice input of the caller includes receiving raw voice data; and
the computer-implemented method further comprises translating the raw voice data to text data.

13. An intelligent voice interface system comprising:
one or more processors; and
one or more memories storing instructions of an intelligent voice interface, wherein the instructions, when executed by the one or more processors, cause the one or more processors to, during a voice communication with a caller via a caller device:
receive, from the caller device, caller input data indicative of a voice input of the caller;
determine, by processing the caller input data, an inferred state of the caller, wherein determining the inferred state of the caller includes analyzing one or more characteristics, other than textual content, of the voice input to determine a caller state score based on a plurality of scores corresponding to a plurality of events exhibiting the one or more characteristics of the voice input, and determine the inferred state in response to determining that the caller state score exceeds a threshold score, wherein the plurality of events are related to two or more of: a speed, a pitch, or a volume of a voice of the caller; and
based upon the inferred state of the caller, select a pathway through an algorithmic dialog that includes one or more available voice prompts for requesting caller information.

14. The intelligent voice interface system of claim 13 , wherein the one or more characteristics include loudness of the voice of the caller, the pitch of the voice of the caller, and/or rapidity with which the caller speaks.

15. The intelligent voice interface system of claim 13 , wherein determining the inferred state of the caller includes determining that the caller is impatient, angry, or frustrated.

16. The intelligent voice interface system of claim 13 , wherein determining the inferred state of the caller includes determining that the caller is happy, content, or satisfied.

17. The intelligent voice interface system of claim 13 , wherein selecting the pathway through the algorithmic dialog includes bypassing one or more voice prompts based upon the inferred state of the caller.

18. The intelligent voice interface system of claim 13 , wherein selecting the pathway through the algorithmic dialog includes:
generating a voice prompt that asks whether the caller would like to be transferred to a human representative; and
sending the voice prompt to the caller device.

19. The intelligent voice interface system of claim 13 , wherein determining the inferred state of the caller includes analyzing (i) the one or more characteristics of voice input and (ii) textual content of the voice input.

20. The intelligent voice interface system of claim 13 , wherein the instructions further cause the one or more processors to:
evaluate the voice communication with the caller based upon the inferred state.

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Marzinzik Duane Lee
• Moore Eric R
• Carter Gregory D
• Lalwani Harsh
• Mifflin Matthew
• Uppaluri Padmaja
• Jewell Ryan
• Lovings Richard J

Classifications

• G10L25/63

  for estimating an emotional state · CPC title

• G06F40/35 Primary

  Discourse or dialogue representation · CPC title

• H04M2201/40

  using speech recognition · CPC title

• G10L15/04

  Segmentation; Word boundary detection · CPC title

• G10L15/30

  Distributed recognition, e.g. in client-server systems, for mobile phones or network applications · CPC title