Natural language understanding processing

Patent metadata

Field	Value
Publication number	US-11335346-B1
Application number	US-201816215061-A
Country	US
Kind code	B1
Filing date	Dec 10, 2018
Priority date	Dec 10, 2018
Publication date	May 17, 2022
Grant date	May 17, 2022

Abstract

Techniques for processing a user input are described. Text data representing a user input is processed with respect to at least one finite state transducer (FST) to generate at least one FST hypothesis. Context information may be required to traverse one or more paths of the at least one FST. The text data is also processed using at least one statistical model (e.g., perform intent classification, named entity recognition, and/or domain classification processing) to generate at least one statistical model hypothesis. The at least one FST hypothesis and the at least one statistical model hypothesis are input to a reranker that determines a most likely interpretation of the user input.

First claim

What is claimed is:

1. A method, comprising:
receiving, from a first device, audio data representing a user input;
performing automatic speech recognition (ASR) processing on the audio data to generate ASR results data;
determining context data including at least a device type of the first device;
generating, using a finite state transducer (FST), a first natural language understanding (NLU) hypothesis including an intent, the first NLU hypothesis corresponding to a first FST path associated with the device type;
performing intent classification (IC) processing with respect to the ASR results data;
performing named entity recognition (NER) processing with respect to the ASR results data;
generating a second NLU hypothesis based at least in part on the IC processing and the NER processing;
processing, using a reranker component, the context data and the first NLU hypothesis to determine a first score;
processing, using the reranker component, the context data and the second NLU hypothesis to determine a second score; and
causing, based at least in part on the first score and the second score, an action to be performed using the first NLU hypothesis.

2. The method of claim 1 , further comprising:
processing, using at least one statistical model, the first NLU hypothesis to determine a third score;
processing, using the at least one statistical model, the second NLU hypothesis to determine a fourth score; and
inputting, to the reranker component, the third score and the fourth score.

3. The method of claim 1 , further comprising:
storing, based at least in part on the first FST path being associated with the device type, an association between the first NLU hypothesis and an indicator representing the first FST path is associated with context data; and
inputting the indicator to the reranker component.

4. The method of claim 1 , further comprising:
determining a user identifier associated with the audio data; and
selecting the FST based at least in part on the FST being associated with the user identifier,
wherein the FST includes a plurality of FST paths including a second FST path, the second FST path being associated with context information represented in user profile data associated with the user identifier.

5. A method, comprising:
receiving first data representing a user input;
determining first context data associated with the user input;
generating, using a finite state transducer (FST), a first natural language understanding (NLU) hypothesis including a first intent corresponding to the user input, the first NLU hypothesis corresponding to a first FST path associated with the first context data;
generating, using at least one statistical model, a second NLU hypothesis including a second intent corresponding to the user input;
determining, using a reranker component, that the first NLU hypothesis is to be used to respond to the user input instead of the second NLU hypothesis; and
causing an action to be performed using the first NLU hypothesis.

6. The method of claim 5 , further comprising:
processing the first NLU hypothesis using the at least one statistical model to determine a first score;
determining, using the at least one statistical model, a second score corresponding to the second NLU hypothesis; and
inputting, to the reranker component, the first score and the second score.

7. The method of claim 6 , further comprising:
determining a domain associated with the first NLU hypothesis;
determining a first statistical model associated with the domain; and
processing, using the first statistical model, the first NLU hypothesis to at least partially generate the first score.

8. The method of claim 5 , further comprising:
generating, using the at least one statistical model, a third NLU hypothesis corresponding to the first NLU hypothesis;
determining a first score associated with the third NLU hypothesis; and
associating the first score with the first NLU hypothesis.

9. The method of claim 5 , further comprising:
associating the first NLU hypothesis with an indicator representing the first FST path is associated with the first context data; and
inputting the indicator to the reranker component.

10. The method of claim 5 , further comprising:
determining a user identifier associated with the first data; and
determining the FST is associated with the user identifier,
wherein the FST includes a plurality of FST paths including a second FST path, the second FST path being associated with second context data represented in user profile data associated with the user identifier.

11. The method of claim 5 , further comprising:
performing intent classification (IC) processing with respect to the first data;
performing named entity recognition (NER) processing with respect to the first data; and
generating the second NLU hypothesis based at least in part on the IC processing and the NER processing.

12. The method of claim 5 , further comprising:
sending, to an NLU entity resolution component, an ordered list output by the reranker component, the ordered list including the first NLU hypothesis and the second NLU hypothesis.

13. A system, comprising:
at least one processor; and
at least one memory comprising instructions that, when executed by the at least one processor, cause the system to:
receive first data representing a user input;
determine first context data associated with the user input;
generate, using a finite state transducer (FST), a first natural language understanding (NLU) hypothesis including a first intent corresponding to the user input, the first NLU hypothesis corresponding to a first FST path associated with the first context data;
generate, using at least one statistical model, a second NLU hypothesis including a second intent corresponding to the user input;
determine, using a reranker component, that the first NLU hypothesis is to be used to respond to the user input instead of the second NLU hypothesis; and
cause an action to be performed using the first NLU hypothesis.

14. The system of claim 13 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
process the first NLU hypothesis using the at least one statistical model to determine a first score;
determine, using the at least one statistical model, a second score corresponding to the second NLU hypothesis; and
input, to the reranker component, the first score and the second score.

15. The system of claim 14 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine a domain associated with the first NLU hypothesis;
determine a first statistical model associated with the domain; and
process, using the first statistical model, the first NLU hypothesis to at least partially generate the first score.

16. The system of claim 13 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
generate, using the at least one statistical model, a third NLU hypothesis corresponding to the first NLU hypothesis;
determine a first score associated with the third NLU hypothesis; and
associate the first score with the first NLU hypothesis.

17. The system of claim 13 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
associate the first NLU hypothesis with a

Assignees

• Amazon Tech Inc

Inventors

• Su Chengwei
• Matsoukas Spyridon
• Ananthakrishnan Sankaranarayanan
• Saleem Shirin
• Chan Chungnam
• Li Yugang
• Mcmanamon Mallory
• Gupta Rahul
• Soldaini Luca

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F40/30

  Semantic analysis · CPC title