Conversational interface for matching text of spoken input based on context model

Patent metadata

Field	Value
Publication number	US-10274911-B2
Application number	US-201514750482-A
Country	US
Kind code	B2
Filing date	Jun 25, 2015
Priority date	Jun 25, 2015
Publication date	Apr 30, 2019
Grant date	Apr 30, 2019

Abstract

Technologies for conversational control of a home automation system include a computing device in communication with one or more home automation devices. The computing device receives text input indicative of a spoken user request including one or more parameters associated with a home automation command. The computing device matches the text input against a home automation language model including several predefined dialog patterns. The computing device resolves parameters of the command using a context model. The computing device may resolve the location, device, or requested device state of the command. The computing device generates a text response and a home automation command using the language model, and then presents the text response to the user and performs the home automation command using the home automation devices. In some embodiments, the computing device may control an application automation system or a data exploration system. Other embodiments are described and claimed.

First claim

The invention claimed is:

1. A computing device for conversational system control, the computing device comprising:
a controller module to receive input text indicative of a spoken user request, wherein the spoken user request includes one or more parameters associated with a system control command for a controlled system;
a conversational interface module to match the input text against a language model to identify a matching pattern, wherein the language model includes a plurality of dialog patterns, wherein the dialog patterns are arranged hierarchically to define one or more dialogs, and wherein each dialog pattern, when evaluated against the input text, determines whether the input text matches the corresponding dialog pattern, and wherein the matching pattern is associated with the system control command; and
a context management module to determine one or more of the parameters of the system control command based on a context model in response to a match of the input text;
wherein the conversational interface module is further to generate the system control command based on the language model in response to a determination of the one or more parameters; and
wherein the controller module is further to perform the system control command to cause a change in system state of the controlled system, wherein the controlled system comprises a home automation system, and wherein the change in the system state comprises a change in device state of a device of the home automation system, wherein the device state comprises an activation state of the device or a set point of the device;
wherein the computing device further comprises a learning module to (i) observe the change in the system state of the controlled system in response to performance of the system control command, (ii) generate a new dialog pattern based on the input text and the change in the device state, wherein the new dialog pattern is associated with the system control command, and (iii) update the language model to include the new dialog pattern.

2. The computing device of claim 1 , wherein:
the matching pattern is further associated with text response;
the conversational interface module is further to generate the text response based on the language model; and
the controller module is further to present the text response to a user of the computing device.

3. The computing device of claim 1 , further comprising:
an automated speech recognition module to perform automated speech recognition to generate the input text; and
a text-to-speech module to generate speech output based on the text response with a text-to-speech engine.

4. The computing device of claim 1 , wherein:
the system control command comprises a home automation command;
the one or more parameters of the home automation command include a location, a device, or a requested device state; and
the context model includes conversational context data, user context data, or home automation device context data.

5. The computing device of claim 4 , wherein to determine the one or more of the parameters of the home automation command based on the context model comprises to determine a location parameter, a device parameter, or a device state parameter of the home automation command based on the conversational context data.

6. The computing device of claim 4 , wherein to determine the one or more of the parameters of the home automation command based on the context model comprises to determine a location parameter of the home automation command based on the user context data.

7. The computing device of claim 4 , wherein to determine the one or more of the parameters of the home automation command based on the context model comprises to determine a device parameter or a device state parameter of the home automation command based on the home automation device context data.

8. The computing device of claim 4 , wherein the context management module is further to update the context model based on data received from the home automation system.

9. The computing device of claim 4 , wherein:
the controller module is further to determine whether the home automation command is valid based on the home automation device context data of the context model in response to the determination of the one or more parameters of the home automation command; and
the conversational interface module is to generate an error message for the text response with the language model in response to a determination that the home automation command is not valid;
wherein to generate the system control command based on the language model comprises to generate the home automation command in response to a determination that the home automation command is valid.

10. A method for conversational system control, the method comprising:
receiving, by a computing device, input text indicative of a spoken user request, wherein the spoken user request includes one or more parameters associated with a system control command for a controlled system;
matching, by the computing device, the input text against a language model to identify a matching pattern, wherein the language model includes a plurality of dialog patterns, wherein the dialog patterns are arranged hierarchically to define one or more dialogs, and wherein each dialog pattern, when evaluated against the input text, determines whether the input text matches the corresponding dialog pattern, and wherein the matching pattern is associated with the system control command;
determining, by the computing device, one or more of the parameters of the system control command using a context model in response to matching the input text;
generating, by the computing device, the system control command using the language model in response to determining the one or more parameters;
performing, by the computing device, the system control command to cause a change in system state of the controlled system, wherein the controlled system comprises a home automation system, and wherein the change in the system state comprises a change in device state of a device of the home automation system, wherein the device state comprises an activation state of the device or a set point of the device;
observing, by the computing device, the change in the system state of the controlled system in response to performing the system control command;
generating, by the computing device, a new dialog pattern based on the input text and the change in the system state, wherein the new dialog pattern is associated with the system control command; and
updating, by the computing device, the language model to include the new dialog pattern.

11. The method of claim 10 , further comprising:
generating, by the computing device, a text response using the language model in response to determining the one or more parameters, wherein the matching pattern is associated with the text response; and
presenting, by the computing device, the text response to a user of the computing device.

12. The method of claim 10 , wherein:
the system control command comprises a home automation command;
the one or more parameters of the home automation command include a location, a device, or a requested device state; and
the context model includes conversational context data, user context data, or home automation device context data.

13. The method of claim 12 , further comprising:
determining, by the computing device, whether the home automation command is valid based on the home automation device context data of the context model in response to determining the one or more parameters of the home automation command; and
generating, by the computing device, an error

Assignees

• Intel Corp

Inventors

• Uppala Anantha Deepthi
• Jackson Kara E
• Yarvis Mark D
• Cobbley David J
• Dickinson Andrew S
• Pandit Milind

Classifications

• G05B2219/2642

  Domotique, domestic, home control, automation, smart house · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• G10L15/183

  using context dependencies, e.g. language models · CPC title

• G10L13/00

  Speech synthesis; Text to speech systems · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title