Hybrid learning system for natural language intent extraction from a dialog utterance

Patent metadata

Field	Value
Publication number	US-10713441-B2
Application number	US-201916238324-A
Country	US
Kind code	B2
Filing date	Jan 2, 2019
Priority date	Mar 23, 2018
Publication date	Jul 14, 2020
Grant date	Jul 14, 2020

Abstract

An agent automation system includes a memory configured to store a natural language understanding (NLU) framework and a processor configured to execute instructions of the NLU framework to cause the agent automation system to perform actions. These actions comprise: generating an annotated utterance tree of an utterance using a combination of rules-based and machine-learning (ML)-based components, wherein a structure of the annotated utterance tree represents a syntactic structure of the utterance, and wherein nodes of the annotated utterance tree include word vectors that represent semantic meanings of words of the utterance; and using the annotated utterance tree as a basis for intent/entity extraction of the utterance.

First claim

What is claimed is:

1. An agent automation system, comprising:
a memory configured to store a natural language understanding (NLU) framework and an intent/entity model that includes written sample utterances; and
a processor configured to execute instructions of the NLU framework to cause the agent automation system to perform actions comprising:
generating annotated utterance trees for a written user utterance and for the written sample utterances of the intent/entity model using a combination of rules-based and machine-learning (ML)-based components, wherein each annotated utterance tree includes nodes arranged in a dependency parse tree structure that represents a syntactic structure of a corresponding utterance, and wherein each of the nodes includes a word vector representing a semantic meaning of a word or phrase of the corresponding utterance;
generating a subtree vector for each subtree of the annotated utterance trees based on the word vectors of the nodes of each subtree of the annotated utterance trees; and
extracting an intent and/or entity from the written user utterance based on a comparison of the subtree vectors of the annotated utterance trees of the written user utterance to the subtree vectors of the annotated utterance trees of the written sample utterances.

2. The agent automation system of claim 1 , wherein the NLU framework includes a vocabulary subsystem, a structure subsystem, and a prosody subsystem that cooperate to generate the annotated utterance tree of the corresponding utterance using the combination of rules-based and machine-learning (ML)-based components.

3. The agent automation system of claim 2 , wherein the vocabulary subsystem includes a word vector distribution model, wherein word vector distribution model provides the word vectors for the nodes of the annotated utterance tree based on the word vector distribution model.

4. The agent automation system of claim 3 , wherein the processor is configured to execute instructions of the NLU framework to cause the agent automation system to improve operation of the vocabulary subsystem by performing actions comprising:
performing rule-based unsupervised learning of the words and/or phrases of the written user utterance; and
modifying word vectors of the word vector distribution model based on the rule-based unsupervised learning.

5. The agent automation system of claim 2 , wherein the structure subsystem includes one or more rules-based parsers and one or more ML-based parsers, wherein the structure subsystem is configured to generate and annotate the structure of the annotated utterance trees based on the syntactic structures of their corresponding utterances.

6. The agent automation system of claim 5 , wherein the processor is configured to execute instructions of the NLU framework to cause the agent automation system to improve operation of the one or more ML-based parsers by performing actions comprising:
parsing the written user utterance using the one or more rules-based parsers of the structure subsystem to generate a plurality of annotated utterance trees; and
in response to the processor determining that a majority of the plurality of annotated utterance trees have a common structure, adjusting at least one weight of one or more neural network models associated with the one or more ML-based parsers such that the one or more ML-based parsers are configured to generate the common structure from the written user utterance.

7. The agent automation system of claim 2 , wherein the prosody subsystem includes one or more rules-based prosody systems and one or more ML-based prosody systems, wherein the prosody system is configured to segment the written user utterance based on written prosody cues of the written user utterance, wherein the written prosody cues comprise a rhythm, an emphasis, or a focus of the written user utterance.

8. The agent automation system of claim 1 , wherein, to extract the intent and/or entity from the written user utterance, the processor is configured to execute instructions of the NLU framework to cause the agent automation system to perform actions comprising:
comparing the subtree vectors of the annotated utterance trees of the written user utterance to the subtree vectors of the annotated utterance trees of the written sample utterances of the intent/entity model to identify a matching written sample utterance;
determining the intent and/or entity of the written user utterance to be a defined intent and/or entity of the intent/entity model that corresponds to the matching written sample utterance within the intent/entity model.

9. The agent automation system of claim 1 , wherein the processor is configured to execute instructions of the NLU framework to cause the agent automation system perform additional actions before generating the subtree vectors, comprising:
performing rule-based error detection to detect, in a particular annotated utterance tree of the annotated utterance trees, a misclassification or misparse of words and/or phrases of the corresponding utterance of the particular annotated utterance tree;
performing a rule-based modification of the corresponding utterance of the particular annotated utterance tree to generate a modified written utterance, wherein at least a portion of words and/or phrases of the modified written utterance are different from the words and/or phrases of the corresponding utterance; and
regenerating the particular annotated utterance tree from the modified written utterance using the combination of rules-based and machine-learning (ML)-based components, wherein the regenerated particular annotated utterance tree lacks the misclassification or the misparse of the particular annotated utterance tree before regeneration.

10. The agent automation system of claim 1 , wherein the intent/entity model associates defined intents with the written sample utterances, and wherein the written sample utterances encode defined entities as parameters of the defined intents within the intent/entity model.

11. A method of operating a natural language understanding (NLU) framework, comprising:
generating an annotated utterance tree for a written user utterance using a combination of rules-based and machine-learning (ML)-based components, wherein the annotated utterance tree includes nodes arranged in a dependency parse tree structure that represents a syntactic structure of the written user utterance, and wherein each of the nodes includes a word vector representing a semantic meaning of a word or phrase of the written user utterance;
performing rule-based error detection of the annotated utterance tree to detect a misclassification or misparse of words and/or phrases of the written user utterance;
performing a rule-based modification of the written user utterance to generate a modified written utterance, wherein at least a portion of words and/or phrases of the modified written utterance are different from the words and/or phrases of the written user utterance;
regenerating the annotated utterance tree from the modified written utterance using the combination of rules-based and machine-learning (ML)-based components; and
using the regenerated annotated utterance tree as a basis for intent/entity extraction of the written user utterance by:
generating a subtree vector for each subtree of the regenerated annotated utterance tree based on the word vectors of the nodes of each subtree of the regenerated annotated utterance trees; and
extracting an intent and/or entity from the written user utterance based on a comparison of the subtree vectors of the regenerated annotated utterance tree to subtree vectors of other annotated utterance trees representing sa

Assignees

• Servicenow Inc

Inventors

• Sapugay Edwin
• Madamala Anil Kumar
• Naboka Maxim
• Sunkara Srinivas Satyasai
• Santos Lewis Savio Landry
• Subbarao Murali B

Classifications

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06N5/022

  Knowledge engineering; Knowledge acquisition · CPC title

• G10L25/48

  specially adapted for particular use · CPC title

• G10L15/19

  Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules · CPC title

• G06N5/025

  Extracting rules from data · CPC title