Processing complex utterances for natural language understanding

Patent metadata

Field	Value
Publication number	US-12462801-B2
Application number	US-202217882874-A
Country	US
Kind code	B2
Filing date	Aug 8, 2022
Priority date	Sep 29, 2016
Publication date	Nov 4, 2025
Grant date	Nov 4, 2025

Abstract

A system capable of performing natural language understanding (NLU) on utterances including complex command structures such as sequential commands (e.g., multiple commands in a single utterance), conditional commands (e.g., commands that are only executed if a condition is satisfied), and/or repetitive commands (e.g., commands that are executed until a condition is satisfied). Audio data may be processed using automatic speech recognition (ASR) techniques to obtain text. The text may then be processed using machine learning models that are trained to parse text of incoming utterances. The models may identify complex utterance structures and may identify what command portions of an utterance go with what conditional statements. Machine learning models may also identify what data is needed to determine when the conditionals are true so the system may cause the commands to be executed (and stopped) at the appropriate times.

First claim

What is claimed is:

1 . A computer-implemented method, comprising:
determining, by a computing system, that an input utterance includes a first portion corresponding to a command and a second portion corresponding to a condition; based at least in part on the command, sending, via a network, a first signal that causes a first component to transition from a first state to a second state, wherein the condition is initially not satisfied when the first component is in the second state; after the first component has transitioned to the second state, determining, by the computing system, that the condition has become satisfied; and based at least in part on the condition having become satisfied, sending, via the network, a second signal that causes the first component to transition from the second state to the first state.

2 . The computer-implemented method of claim 1 , further comprising:
receiving, from at least one second component, first data, wherein determining that the condition has become satisfied is based at least in part on the first data.

3 . The computer-implemented method of claim 2 , further comprising:
determining that the condition corresponds to a first data type; and based at least on part on the condition corresponding to the first data type, determining to use the first data to determine satisfaction of the condition.

4 . The computer-implemented method of claim 2 , wherein the at least one second component comprises a clock and the first data comprises time data.

5 . The computer-implemented method of claim 2 , wherein the at least one second component comprises an appliance and the first data represents an operational state of the appliance.

6 . The computer-implemented method of claim 2 , wherein the at least one second component comprises a thermometer and the first data comprises temperature data.

7 . The computer-implemented method of claim 2 , wherein the at least one second component comprises a light bulb and the first data represents an operational state of the light bulb.

8 . The computer-implemented method of claim 2 , wherein the at least one second component comprises a global positioning system (GPS) sensor and the first data comprises location data.

9 . The computer-implemented method of claim 1 , wherein the first component comprises a thermostat.

10 . The computer-implemented method of claim 1 , wherein the first component comprises a music player.

11 . The computer-implemented method of claim 1 , wherein first component comprises a light bulb.

12 . A computing system, comprising:
at least one processor; and at least one computer-readable medium encoded with instructions that, when executed by the at least one processor, cause the computing system to:
determine that an input utterance includes a first portion corresponding to a command and a second portion corresponding to a condition;
based at least in part on the command, send, via a network, a first signal that causes a first component to transition from a first state to a second state, wherein the condition is initially not satisfied when the first component is in the second state;
after the first component has transitioned to the second state, determine that the condition has become satisfied; and
based at least in part on the condition having become satisfied, send, via the network, a second signal that causes the first component to transition from the second state to the first state.

13 . The computing system of claim 12 , wherein the at least one computer-readable medium is further encoded with additional instructions which, when executed by the at least one processor, further cause the computing system to:
receive, from at least one second component, first data; and determine that the condition has become satisfied based at least in part on the first data.

14 . The computing system of claim 13 , wherein the at least one computer-readable medium is further encoded with additional instructions which, when executed by the at least one processor, further cause the computing system to:
determine that the condition corresponds to a first data type; and based at least on part on the condition corresponding to the first data type, determine to use the first data to determine satisfaction of the condition.

15 . The computing system of claim 13 , wherein the at least one second component comprises a clock and the first data comprises time data.

16 . The computing system of claim 13 , wherein the at least one second component comprises a household appliance and the first data represents an operational state of the household appliance.

17 . The computing system of claim 13 , wherein the at least one second component comprises a thermometer and the first data comprises temperature data.

18 . The computing system of claim 13 , wherein the at least one second component comprises a light bulb and the first data represents an operational state of the light bulb.

19 . The computing system of claim 13 , wherein the at least one second component comprises a global positioning system (GPS) sensor and the first data comprises location data.

20 . The computing system of claim 12 , wherein the first component comprises a thermostat.

21 . The computing system of claim 12 , wherein the first component comprises a music player.

22 . The computing system of claim 12 , wherein the first component comprises a light bulb.

Assignees

• Amazon Tech Inc

Inventors

• Erbas Cengiz
• Kollar Thomas
• Sikka Avnish
• Matsoukas Spyridon
• Reavely Simon Peter

Classifications

• G10L15/1822

  Parsing for meaning understanding · CPC title

• G10L15/02

  Feature extraction for speech recognition; Selection of recognition unit · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/063

  Training · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title