Ambient assistant device

Patent metadata

Field	Value
Publication number	US-10210866-B2
Application number	US-201715599402-A
Country	US
Kind code	B2
Filing date	May 18, 2017
Priority date	Jan 20, 2017
Publication date	Feb 19, 2019
Grant date	Feb 19, 2019

Abstract

Ambient assistance is described. An assistant device can detect speech in its environment and determine that the speech includes words or phrases of a local dictionary of the assistant device. The assistant device can then generate an interaction opportunity based on the words or phrases of the speech.

First claim

We claim:

1. A home assistant device, comprising:
a speaker;
a microphone;
a display screen;
one or more processors; and
memory storing instructions, wherein the processor is configured to execute the instructions such that the processor and memory are configured to:
detect first speech spoken within an environment of the home assistant device using the microphone;
determine that the first speech includes content having one or more words or phrases included in a local dictionary of the home assistant device;
provide a first interaction opportunity with the home assistant device based on the one or more words or phrases of the first speech corresponding to the local dictionary, the first interaction opportunity providing a speech response using the speaker based on the content of the first speech and based on the first speech including the content having the one or more words or phrases included in the local dictionary of the home assistant device;
detect second speech spoken within the environment of the home assistant device, the first speech being different than the second speech;
determine that the content of the second speech does not include the one or more words or phrases corresponding to the local dictionary;
provide the second speech to a cloud server to determine content related to the second speech;
receive response data from the cloud server based on the second speech; and
provide a second interaction opportunity with the home assistant device based on the response data received from the cloud server, the second interaction opportunity different than the first interaction opportunity, the second interaction opportunity providing a visual response on the display screen based on the content of the second speech and based on the second speech being provided to the cloud server.

2. A method, comprising:
detecting first speech spoken within an environment of an assistant device;
determining, by a processor, that the first speech includes content having one or more words or phrases corresponding to a local dictionary of the assistant device;
generating a first interaction opportunity with the assistant device based on the one or more words or phrases of the first speech corresponding to the local dictionary, the first interaction opportunity being a first type of interaction opportunity based on the first speech including content having the one or more words or phrases corresponding to the local dictionary of the assistant device;
detecting second speech spoken within the environment of the assistant device, the first speech being different than the second speech;
determining that the second speech does not include the one or more words or phrases corresponding to the local dictionary;
providing the second speech to a cloud server to determine content or interactions related to the second speech;
receiving response data from the cloud server based on the content of the second speech; and
generating a second interaction opportunity with the assistant device based on the response data received from the cloud server, the second interaction opportunity being a second type of interaction opportunity that is different than the first type of interaction opportunity, the second interaction opportunity being the second type based on providing the second speech to the cloud server.

3. The method of claim 2 , wherein the cloud server is selected from among a first cloud server and a second cloud server based on the second speech, the first cloud server and the second cloud server corresponding to different services.

4. The method of claim 3 , wherein the cloud server is selected based on characteristics of the second speech, the characteristics including one or more of time, content, complexity, or time duration.

5. The method of claim 2 , wherein the first interaction opportunity includes providing additional information related to content of the first speech.

6. The method of claim 2 , wherein the local dictionary includes information related to translating portions of the first speech into text.

7. The method of claim 2 , wherein the local dictionary includes information related to commands capable of being performed by the assistant device.

8. The method of claim 2 , wherein the first interaction opportunity is a speech response responsive to the first speech, and the second interaction opportunity is a visual response responsive to the second speech.

9. An assistant device, comprising:
one or more processors; and
memory storing instructions, wherein the processor is configured to execute the instructions such that the processor and memory are configured to:
detect first speech spoken within an environment of the assistant device;
determine that the first speech includes content having one or more words or phrases corresponding to a local dictionary of the assistant device;
generate a first interaction opportunity with the assistant device based on the one or more words or phrases of the first speech corresponding to the local dictionary, the first interaction opportunity being a first type of interaction opportunity based on the first speech including content having the one or more words or phrases corresponding to the local dictionary of the assistant device;
detect second speech spoken within the environment of the assistant device, the first speech being different than the second speech;
determine that the second speech does not include the one or more words or phrases corresponding to the local dictionary;
provide the second speech to a cloud server to determine content or interactions related to the second speech;
receive response data from the cloud server based on the content of the second speech; and
generate a second interaction opportunity with the assistant device based on the response data received from the cloud server, the second interaction opportunity being a second type of interaction opportunity that is different than the first type of interaction opportunity, the second interaction opportunity being the second type based on providing the second speech to the cloud server.

10. The assistant device of claim 9 , wherein the cloud server is selected from among a first cloud server and a second cloud server based on the second speech, the first cloud server and the second cloud server corresponding to different services.

11. The assistant device of claim 10 , wherein the cloud server is selected based on characteristics of the second speech, the characteristics including one or more of time, content, complexity, or time duration.

12. The assistant device of claim 9 , wherein the first interaction opportunity includes providing additional information related to content of the first speech.

13. The assistant device of claim 9 , wherein the local dictionary includes information related to translating portions of the first speech into text.

14. The assistant device of claim 9 , wherein the local dictionary includes information related to commands capable of being performed by the assistant device.

15. The assistant device of claim 9 , wherein the first interaction opportunity is a speech response responsive to the first speech, and the second interaction opportunity is a visual response responsive to the second speech.

16. A computer program product, comprising one or more non-transitory computer-readable media having computer program instructions stored therein, the computer program instructions being configured such that, when executed by one or more computing devices, the computer program instructions cause the one or more computing devices to:
detect

Assignees

• Essential Products Inc

Inventors

• Segal Mara Clair
• Roman Manuel
• Desai Dwipal
• Rubin Andrew E

Classifications

• G10L15/30

  Distributed recognition, e.g. in client-server systems, for mobile phones or network applications · CPC title

• G10L15/24

  Speech recognition using non-acoustical features · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/08

  Speech classification or search · CPC title

• G10L2015/088

  Word spotting · CPC title