Systems and method for vocabulary management in a natural learning framework

Patent metadata

Field	Value
Publication number	US-11681877-B2
Application number	US-202117249759-A
Country	US
Kind code	B2
Filing date	Mar 11, 2021
Priority date	Mar 23, 2018
Publication date	Jun 20, 2023
Grant date	Jun 20, 2023

Abstract

An agent automation system implements a virtual agent that is capable of learning new words, or new meanings for known words, based on exchanges between the virtual agent and a user in order to customize the vocabulary of the virtual agent to the needs of the user or users. The agent automation framework has access to a corpus of previous exchanges between the virtual agent and the user, such as one or more chat logs. New words and/or new meanings for known words are identified within the corpus and new word vectors are generated for these new words and/or new meanings for known words and added to refine a word vector distribution model. The refined word vector distribution model is then utilized by the agent automation system to interact with the user.

First claim

What is claimed is:

1. A system, comprising:
a memory configured to store a natural language understanding (NLU) framework, wherein the NLU framework includes a vocabulary model and a prosody subsystem;
a processor configured to execute instructions to cause the NLU framework to perform actions comprising:
receiving a user utterance;
applying the prosody subsystem to segment the user utterance into a plurality of phrases based on written prosody cues of the user utterance, wherein the written prosody cues comprise rhythm, emphasis, or linguistic style within the user utterance; and
for each phrase of the plurality of phrases:
determining whether context is available for the phrase;
in response to determining that context is available for the phrase, performing context-based disambiguation of the phrase and then attempting to determine a respective vector for the disambiguated phrase in the vocabulary model;
in response to determining that context is not available for the phrase, attempting to determine the respective vector for the phrase in the vocabulary model based on a surface form of the phrase; and
in response to determining that the respective vector has not been located for the phrase, applying a null word rule to attempt to determine the respective vector for the phrase.

2. The system of claim 1 , wherein a particular phrase of the plurality of phrases includes a single word of the user utterance, and the respective vector of the particular phrase is a word vector that represents the single word in a semantic vector space of the vocabulary model.

3. The system of claim 1 , wherein a particular phrase of the plurality of phrases includes a plurality of words of the user utterance, wherein determining the respective vector comprises applying a multi-vector aggregation algorithm of the vocabulary model, and wherein the respective vector of the particular phrase is a subphrase vector that is a multi-vector aggregation of word vectors representing the plurality of words in a semantic vector space of the vocabulary model.

4. The system of claim 3 , wherein the multi-vector aggregation algorithm aggregates the word vectors using focus/attention/magnification (FAM) coefficients of the NLU framework.

5. The system of claim 1 , wherein performing the context-based disambiguation of the phrase comprises applying context processing rules of the NLU framework.

6. The system of claim 1 , wherein performing the context-based disambiguation of the phrase comprises applying a structure service and an ontology service of the NLU framework, wherein the structure service is configured to extract a linguistic structure of the phrase and the ontology service is configured to access a lexical database to determine a disambiguated form of the phrase based on the extracted linguistic structure.

7. The system of claim 1 , wherein applying the null word rule comprises determining the respective vector for the phrase by aggregating word vectors of words surrounding the phrase in the user utterance.

8. The system of claim 1 , wherein applying the null word rule comprises using a trained machine-learning (ML) model to determine the respective vector from an ordered collection of characters of the phrase.

9. A method of operating a natural language understanding (NLU) framework, comprising:
receiving a user utterance;
applying a prosody subsystem of the NLU framework to segment the user utterance into a plurality of phrases based on written prosody cues of the user utterance; and
for each phrase of the plurality of phrases:
determining whether context is available for the phrase;
in response to determining that context is available for the phrase, performing context-based disambiguation of the phrase and then attempting to determine a respective vector for the disambiguated phrase in a vocabulary model of the NLU framework;
in response to determining that context is not available for the phrase, attempting to determine the respective vector for the phrase in the vocabulary model based on a surface form of the phrase; and
in response to determining that the respective vector has not been located for the phrase, applying a null word rule to attempt to determine the respective vector for the phrase;
wherein determining the respective vector for the phrase or the disambiguated phrase in the vocabulary model comprises applying a multi-vector aggregation algorithm of the vocabulary model to aggregate word vectors representing words of the phrase or the disambiguated phrase into a subphrase vector.

10. The method of claim 9 , wherein the subphrase vector is a multi-vector aggregation of the word vectors, and wherein the multi-vector aggregation algorithm aggregates the word vectors using focus/attention/magnification (FAM) coefficients.

11. The method of claim 9 , wherein performing the context-based disambiguation of the phrase comprises applying context processing rules of the NLU framework, wherein the context processing rules comprise a rule that determines a part of speech of a portion of the phrase.

12. The method of claim 9 , wherein the written prosody cues comprise rhythm, emphasis, linguistic style, or a combination thereof, within the user utterance.

13. The method of claim 9 , wherein applying the null word rule comprises determining the respective vector for the phrase by aggregating word vectors of words surrounding the phrase in the user utterance.

14. The method of claim 9 , wherein applying the null word rule comprises using a trained machine-learning (ML) model to determine the respective vector for the phrase, wherein the ML model is configured and trained to receive an ordered collection of characters of the phrase as input and to provide the respective vector for the phrase as output.

15. A non-transitory, computer-readable medium storing instructions of a natural language understanding (NLU) framework that are executable by one or more processors of a computing system, wherein the instructions comprise instructions to:
receive a user utterance;
apply a prosody subsystem of the NLU framework to segment the user utterance into a plurality of phrases based on written prosody cues of the user utterance; and
for each phrase of the plurality of phrases:
determine whether context is available for the phrase;
in response to determining that context is available for the phrase, perform context-based disambiguation of the phrase using a structure service and an ontology service of the NLU framework and then attempt to determine a respective vector for the disambiguated phrase in a vocabulary model of the NLU framework, wherein the structure service is configured to extract a linguistic structure of the phrase and the ontology service is configured to access a lexical database to determine a disambiguated form of the phrase based on the extracted linguistic structure;
in response to determining that context is not available for the phrase, attempt to determine the respective vector for the phrase in the vocabulary model based on a surface form of the phrase; and
in response to determining that the respective vector has not been located for the phrase, apply a null word rule to attempt to determine the respective vector for the phrase.

16. The medium of claim 15 , wherein a particular phrase of the plurality of phrases includes a plurality of words of the user utterance, wherein the instructions to determine the respective vector comprise instructions to apply a multi-vector aggregation algorithm of the vocabulary model, wherein the respective vector of the particular phrase is a subphrase vec

Assignees

• Servicenow Inc

Inventors

• Sapugay Edwin
• Madamala Anil Kumar
• Naboka Maxim
• Sunkara Srinivas Satyasai
• Santos Lewis Savio Landry
• Subbarao Murali B

Classifications

• G06F40/205

  Parsing · CPC title

• G10L15/1807

  using prosody or stress · CPC title

• G10L2015/225

  Feedback of the input speech · CPC title

• G06N5/022

  Knowledge engineering; Knowledge acquisition · CPC title

• G06N20/00

  Machine learning · CPC title