Assistant device with active and passive experience modes

Patent metadata

Field	Value
Publication number	US-10102856-B2
Application number	US-201715599407-A
Country	US
Kind code	B2
Filing date	May 18, 2017
Priority date	Jan 20, 2017
Publication date	Oct 16, 2018
Grant date	Oct 16, 2018

Abstract

Ambient assistance is described. An assistant device can operate in an active experience mode in which a response to a user's speech is provided if it includes a hardware activation phrase. Based on characteristics of the environment, the mode can be adjusted to a passive experience mode in which a response to speech is provided even if it does not include the hardware activation phrase.

First claim

We claim:

1. A home assistant device, comprising:
a camera;
one or more processors; and
memory storing instructions, wherein the processor is configured to execute the instructions such that the processor and memory are configured to:
operate the home assistant device in a passive experience mode, the passive experience mode corresponding to the home assistant device providing a first response to a first speech based on the first speech not including a hardware activation phrase;
determine characteristics of an environment of the home assistant device using the camera, the characteristics of the environment including determining that a user is engaged with playback of media content in the environment; and
switch operation of the home assistant device from the passive experience mode to an active experience mode based on the determined characteristics including that the user is engaged with the playback of the media content, the active experience mode corresponding to the home assistant device providing a second response to a second speech, the second speech including the hardware activation phrase.

2. An electronic device, comprising:
one or more processors; and
memory storing instructions, wherein the processor is configured to execute the instructions such that the processor and memory are configured to:
operate the electronic device in a passive experience mode providing a first response to first speech based on the first speech not including a hardware activation phrase;
determine characteristics of an environment of the electronic device, the characteristics of the environment including a determination that a user is engaged with playback of media content; and
adjust operation of the electronic device from the passive experience mode to an active experience mode based on the characteristics including that the user is engaged with the playback of the media content, the active experience mode configured to provide a second response to second speech including the hardware activation phrase.

3. The electronic device of claim 2 , wherein the hardware activation phrase indicates an intention of the user providing the second speech to interact with the electronic device.

4. The electronic device of claim 2 , wherein the electronic device further includes a camera and the characteristics of the environment is based on visual input provided by the camera.

5. The electronic device of claim 2 , wherein the processor is configured to execute the instructions such that the processor and memory are configured to:
determine characteristics of the second speech, wherein the adjusting operation of the electronic device from the passive experience mode to the active experience mode is further based on the characteristics of the second speech.

6. The electronic device of claim 2 , wherein the characteristics of the environment include a number of people within the environment.

7. A method, comprising:
operating an electronic device in a passive experience mode, the passive experience mode configured to provide a first response to a first speech not including a hardware activation phrase;
determining, by a processor, characteristics of an environment of the electronic device the characteristics of the environment including a determination that a user is involved in activity related to playback of media content in the environment; and
adjusting operation of the electronic device from the passive experience mode to an active experience mode based on the characteristics including that the user is involved in activity related to the playback of the media content in the environment, the active experience mode providing a second response to second speech based on the second speech including the hardware activation phrase.

8. The method of claim 7 , wherein the hardware activation phrase indicates an intention of a user providing the second speech to interact with the electronic device.

9. The method of claim 7 , wherein the electronic devices further includes a camera and the characteristics of the environment is based on visual input provided by the camera.

10. The method of claim 9 , wherein the characteristics of the environment indicate that the user is looking at or towards the electronic device.

11. The method of claim 7 , further comprising:
determining characteristics of the second speech, wherein the adjusting operation of the electronic device from the passive experience mode to the active experience mode is further based on the characteristics of the second speech.

12. The method of claim 7 , wherein the characteristics of the environment include a number of people within the environment.

13. The method of claim 7 , further comprising:
receiving information from a display device regarding playback of media content in the environment; and
adjusting operation of the electronic device from the active experience mode to the passive experience mode based on the information regarding the playback of the media content in the environment.

14. A computer program product, comprising one or more non-transitory computer-readable media having computer program instructions stored therein, the computer program instructions being configured such that, when executed by one or more computing devices, the computer program instructions cause the one or more computing devices to:
operate the computing device in a passive experience mode providing a first response to first speech based on the first speech not including a hardware activation phrase;
determine characteristics of an environment of the computing device, the characteristics of the environment including a determination that a user is engaged with playback of media content; and
adjust operation of the computing device from the passive experience mode to an active experience mode based on the characteristics including that the user is engaged with playback of media content, the active experience mode configured to provide a second response to second speech including the hardware activation phrase.

15. The computer program product of claim 14 , wherein the hardware activation phrase indicates an intention of the user providing the second speech to interact with the computing device.

16. The computer program product of claim 14 , wherein the computing device further includes a camera and the characteristics of the environment is based on visual input provided by the camera.

17. The computer program product of claim 16 , wherein the characteristics of the environment indicate that the user is looking at or towards the computing device.

18. The computer program product of claim 14 , the computer program instructions cause the one or more computing devices to:
determine characteristics of the second speech, wherein the adjusting operation of the computing device from the passive experience mode to the active experience mode is further based on the characteristics of the second speech.

19. The computer program product of claim 14 , wherein the characteristics of the environment include a number of people within the environment.

Assignees

• Essential Products Inc

Inventors

• Segal Mara Clair
• Roman Manuel
• Desai Dwipal
• Rubin Andrew E

Classifications

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L2015/226

  using non-speech characteristics · CPC title

• G10L15/08

  Speech classification or search · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L15/30

  Distributed recognition, e.g. in client-server systems, for mobile phones or network applications · CPC title