Routing of cases using unstructured input and natural language processing

Patent metadata

Field	Value
Publication number	US-10997250-B2
Application number	US-201816140418-A
Country	US
Kind code	B2
Filing date	Sep 24, 2018
Priority date	Sep 24, 2018
Publication date	May 4, 2021
Grant date	May 4, 2021

Abstract

A method is provided for generating a ranked list of candidate responders. In some embodiments, the method includes receiving a question from a user and generating a question feature vector representing an intent of the question and a first skill set inferred from the question. The method also includes for one or more candidate responders, generating a candidate feature vector representing a skill set and questions associated with the respective candidate responder; computing a reputation score based on questions and user feedback associated with the respective candidate responder; and computing, based on the question feature vector, candidate feature vector, and reputation score, a probability score representing a prediction of the quality of an answer that would be provided by the respective candidate responder if the input question were routed to the respective candidate responder. The method further includes generating a ranked list of candidate responders using the computed probability scores.

First claim

What is claimed is:

1. A computing device comprising:
a memory containing machine readable medium storing machine executable code; and
one or more processors coupled to the memory and configurable to execute the machine executable code to cause the one or more processors to:
receive a text input question from a user;
generate a question feature vector representing an intent of the input question and a first skill set inferred from the input question;
for each of one or more candidate responders:
generate a candidate feature vector representing a skill set of the respective candidate responder, a first set of input questions previously answered by the respective candidate responder, and metadata profile information provided by the respective candidate responder, the metadata profile information including one or more candidate skill sets and one or more candidate skill levels corresponding to the one or more candidate skill sets;
compute a reputation score based on a second set of input questions previously answered by the respective candidate responder and user feedback for the respective candidate responder using a set of weights, wherein the weights are provided by domain experts or are learned in a training process; and
compute, based on the question feature vector, the candidate feature vector, and the reputation score, a probability score representing a prediction of a quality of an answer that would be provided by the respective candidate responder to the input question if the input question were routed to the respective candidate responder; and

generate a ranked list of candidate responders using the probability score.

2. The computing device of claim 1 , wherein the question feature vector represents a topic of the input question.

3. The computing device of claim 1 , wherein the candidate feature vector represents inferred topics derived from a candidate responder's posts.

4. The computing device of claim 1 , wherein the machine executable code further causes the one or more processors to:
for each of one or more candidate responders, collect activity information for the respective candidate responder, wherein the probability score is computed based on the activity information.

5. The computing device of claim 4 , wherein the activity information includes at least one of the candidate responder's average lag time in responding to a set of questions, posting activity and frequency, or login activity.

6. The computing device of claim 1 , wherein the skill set of the respective candidate responder is derived from the respective candidate responder's profile and an answer to at least one of the first set of input questions.

7. The computing device of claim 1 , wherein the machine executable code further causes the one or more processors to:
receive user feedback indicating a quality of one or more answers previously provided by a candidate responder to one or more questions, wherein the reputation score is based on the user feedback.

8. The computing device of claim 7 , wherein the machine executable code further causes the one or more processors to:
receive an indication that a user has selected a user-selectable option that when selected provides the user feedback.

9. The computing device of claim 1 , wherein the machine executable code further causes the one or more processors to:
send a notification of the input question to the first listed candidate responder in the ranked list of candidate responders.

10. The computing device of claim 9 , wherein the machine executable code further causes the one or more processors to:
identify a schedule for sending notifications to candidate responders, wherein the notification is sent in accordance with the schedule.

11. The computing device of claim 9 , wherein the machine executable code further causes the one or more processors to:
determine that a time period has elapsed before the first candidate responder has responded to the input question; and
in response to a determination that the time period has elapsed, send the notification to the second listed candidate responder in the ranked list of candidate responders.

12. A method performed by one or more processors executing machine executable code, the method comprising:
receiving a text input question from a user;
generating a question feature vector representing an intent of the input question and a first skill set inferred from the input question;
for each of one or more candidate responders:
generating a candidate feature vector representing a skill set of the respective candidate responder, a first set of input questions previously answered by the respective candidate responder, and metadata profile information provided by the respective candidate responder, the metadata profile information including one or more candidate skill sets and one or more candidate skill levels corresponding to the one or more candidate skill sets;
computing a reputation score based on a second set of input questions previously answered by the respective candidate responder and user feedback for the respective candidate responder using a set of weights, wherein the weights are provided by domain experts or are learned in a training process; and
computing, based on the question feature vector, the candidate feature vector, and the reputation score, a probability score representing a prediction of a quality of an answer that would be provided by the respective candidate responder to the input question if the input question were routed to the respective candidate responder; and

generating a ranked list of candidate responders using the probability score.

13. The method of claim 12 , wherein the candidate feature vector represents an answer to at least one question of the first set of questions.

14. The method of claim 12 , wherein a reputation score is further based on an answer to at least one question of the second set of questions.

15. The method of claim 12 , further comprising:
receiving user feedback indicating a quality of one or more answers previously provided by a candidate responder to one or more questions, wherein the reputation score is based on the user feedback.

16. The method of claim 12 , further comprising:
sending a notification of the input question to the first listed candidate responder in the ranked list of candidate responders.

17. The method of claim 16 , wherein the notification includes an incentive for the first listed candidate responder to respond to the input question.

18. A non-transitory machine-readable medium comprising executable code which when executed by one or more processors associated with a computing device are adapted to cause the one or more processors to perform a method comprising:
receiving a text input question from a user;
generating a question feature vector representing an intent of the input question and a first skill set inferred from the input question;
for each of one or more candidate responders:
generating a candidate feature vector representing a skill set of the respective candidate responder, a first set of input questions previously answered by the respective candidate responder, and metadata profile information provided by the respective candidate responder, the metadata profile information including one or more candidate skill sets and one or more candidate skill levels corresponding to the one or more candidate skill sets;
computing a reputation score based on a second set of input questions previously answered by the respective candidate responder and user feedback for the

Assignees

• Salesforce Com Inc

Inventors

• Asur Sitaram
• Sakhuja Aditya
• Fisher Hui S
• Goswami Anjan
• Le Khoa

Classifications

• G06F9/453 Primary

  Help systems · CPC title

• G06F40/295

  Named entity recognition · CPC title

• G06F16/90332 Primary

  Natural language query formulation or dialogue systems · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G06F16/3329

  Natural language query formulation · CPC title