System and method for customizing portable natural language processing interface for appliances

Patent metadata

Field	Value
Publication number	US-10978046-B2
Application number	US-201816160934-A
Country	US
Kind code	B2
Filing date	Oct 15, 2018
Priority date	Oct 15, 2018
Publication date	Apr 13, 2021
Grant date	Apr 13, 2021

Abstract

A method and system of customizing a portable voice-based control user interface for multiple types of appliances are disclosed. The method performed at a user device includes establishing a data communication connection with a voice control apparatus; detecting a first user request to update a NLP module of the voice control apparatus; establishing a connection to a NLP model server; displaying a listing of appliance types and a respective listing of appliance functions for each appliance type, in a graphical user interface of the user device; receiving user selection of a first set of appliance functions for a first appliance type and a second set of appliance functions for a second appliance type; downloading, from the NLP model server, a first NLP model for the first set of appliance functions for the first appliance type, and a second NLP model for the second set of appliance functions for the second appliance type; and integrating the first NLP model and second NLP model into the NLP module of the voice control apparatus.

First claim

What is claimed is:

1. A method of customizing a portable voice-based control user interface for multiple types of appliances, comprising:
at a user device having one or more processors, memory, and a display:
establishing a data communication connection with a voice control apparatus, the voice control apparatus including a built-in data communication interface that is configured to establish data communication with multiple types of appliances configured to respond to distinct sets of machine commands to perform their respective appliance operations, a built-in voice communication interface that is configured to accept voice-based inputs from a user, and a built-in natural-language processing (NLP) module stored in the memory of the voice control apparatus, wherein the NLP module is configured to store multiple NLP models and selectively utilize a respective one of the multiple NLP models in accordance with a specified target appliance for a currently received voice input, wherein each of the multiple NLP models is trained on respective voice commands for performing a respective set of appliance functions for a respective appliance type;
detecting a first user request to update the NLP module of the voice control apparatus; and
in response to detecting the first user request to update the NLP module of the voice control apparatus:
establishing a connection to a NLP model server corresponding to the voice control apparatus;
displaying, on the display, a listing of appliance types including a first appliance type and a second appliance type, and a respective listing of appliance functions for each appliance type, in a graphical user interface, wherein the first appliance type is configured to perform a first set of appliance functions and the second appliance type is configured to perform a second set of appliance functions different from the first set of appliance functions;
receiving user selection of a first subset of the first set of appliance functions for a first appliance type and a second subset of the second set of appliance functions for a second appliance type, displayed in the graphical user interface; and
in response to receiving the user selection of the first subset of the first set of appliance functions for the first appliance type and the second subset of the second set of appliance functions for the second appliance type:
downloading, from the NLP model server, a first NLP model that is trained on voice commands for the first subset of the first set of appliance functions for the first appliance type, and a second NLP model that is trained on voice commands for the second subset of the second set of appliance functions for the second appliance type; and
integrating the downloaded first NLP model and second NLP model into the NLP module of the voice control apparatus.

2. The method of claim 1 , wherein establishing the data communication connection with the voice control apparatus comprises:
receiving a data communication request from the voice control apparatus when the voice control apparatus is charging a battery of the voice control apparatus.

3. The method of claim 1 , wherein the first user request is detected in response to connecting the voice control apparatus to the user device.

4. The method of claim 1 , wherein detecting the first user request to update the NLP module of the voice control apparatus comprises:
detecting a user input for launching a user application for managing the voice control apparatus in the user device; and
detecting a user selection of an update function displayed in a graphical user interface of the user application.

5. The method of claim 1 , further comprising:
after integrating the first NLP model into the NLP model module at the voice control apparatus:
establishing the data communication connection with the voice control apparatus;
receiving, from the voice control apparatus, voice data of one or more voice commands from the user for controlling the first subset of the first set of appliance functions for the first appliance type;
uploading the voice data to the NLP model server corresponding to the voice control apparatus after establishing the connection to the NLP model server;
receiving the first NLP model from the NLP model server after the NLP model server updates the first NLP model using the voice data of the one or more voice commands; and
integrating the updated first NLP model into the NLP module of the voice control apparatus.

6. The method of claim 1 , further comprising:
after integrating the first NLP model into the NLP model module at the voice control apparatus:
establishing the data communication connection with the voice control apparatus;
receiving, from the voice control apparatus, voice data of one or more voice commands from the user for controlling the first subset of the first set of appliance functions for the first appliance type;
adjusting the first NLP model based on the voice data of the one or more voice commands at the user device; and
integrating the adjusted first NLP model into the NLP module of the voice control apparatus.

7. The method of claim 1 , further comprising:
after integrating the first NLP model into the NLP model module at the voice control apparatus:
establishing the data communication connection with the voice control apparatus;
receiving a first user input to unselect the first subset of the first set of appliance functions for the first appliance type from the listing of appliance types and the respective listing of appliance functions for each appliance type displayed in the graphical user interface, wherein the first NLP model corresponding to the first subset of the first set of appliance functions for the first appliance type has been integrated into the NLP module of the voice control apparatus; and
in response to the first user input, sending a deleting command to the voice control apparatus to remove the first NLP model from the NLP module of the voice control apparatus.

8. The method of claim 1 , further comprising:
after integrating the first NLP model into the NLP model module at the voice control apparatus:
establishing the data communication connection with the voice control apparatus;
receiving a second user input that unselects the first subset of the first set of appliance functions for the first appliance type from the listing of appliance types and the respective listing of appliance functions for each appliance type displayed in the graphical user interface, wherein the first NLP model corresponding to the first subset of the first set of appliance functions for the first appliance type has been integrated into the NLP module of the voice control apparatus;
sending a request to the NLP model server to adjust the first NLP model based on the removal of the first subset of the first set of appliance functions for the first appliance type;
downloading the adjusted first NLP model that is updated to exclude the first subset of the first set of appliance functions for the first appliance type; and
integrating the adjusted first NLP model into the NLP module of the voice control apparatus.

9. The method of claim 1 , further comprising:
after integrating the first NLP model into the NLP model module at the voice control apparatus:
establishing the data communication connection with the voice control apparatus;
receiving a second user input to unselect the first subset of the first set of appliance functions for the first appliance type from the listing of appliance types and the respective listing of appliance functions for each appliance type displayed in the graphical user interface, wherein the first NLP model corresponding to the first subset of the first set of a

Assignees

• Midea Group Co Ltd

Inventors

• Huang Haibin
• Zhang Chen
• Liu Xin

Classifications

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/063 Primary

  Training · CPC title

• G10L15/1815

  Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning · CPC title

• G10L15/30

  Distributed recognition, e.g. in client-server systems, for mobile phones or network applications · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title