Systems, methods, and devices for activity monitoring via a home assistant

Patent metadata

Field	Value
Publication number	US-11830333-B2
Application number	US-202217964183-A
Country	US
Kind code	B2
Filing date	Oct 12, 2022
Priority date	May 12, 2017
Publication date	Nov 28, 2023
Grant date	Nov 28, 2023

Abstract

The various implementations described herein include methods, devices, and systems for monitoring activity in a home environment. In one aspect, a method performed at a voice-assistant device includes: transitioning to operating in a second mode from a first mode; while operating in the second mode, detecting a sound; obtaining a determination as to whether the sound meets one or more monitoring criteria; and in accordance with a determination that the sound meets the one or more monitoring criteria, generating an alert or notification.

First claim

What is claimed is:

1. A method for monitoring activity in a home environment, comprising:
at a computer system having a voice-activated user interface, one or more microphones, one or more speakers, one or more processors, and memory, wherein the computer system has at least two operating modes, including a first mode and a second mode:
while operating in the first mode, receive a voice command, wherein the first mode is an assistant mode; and
while operating in the first mode, output a spoken response to the voice command;
transitioning, by the computer system, from operating in the first mode to operating in the second mode, wherein the second mode is a monitoring mode;
while operating in the second mode, detecting, by the computer system, a sound;
while operating in the second mode, obtaining, by the computer system, a determination that the sound meets one or more monitoring criteria; and
in accordance with the determination that the sound meets the one or more monitoring criteria and while operating in the second mode, generating an alert or notification.

2. The method of claim 1 , wherein the voice command requests the computer system operate in the second mode.

3. The method of claim 1 , wherein the alert or notification would not be generated in the first mode.

4. The method of claim 1 , wherein generating the alert or notification includes emitting a response a simulated human response or a simulated animal response.

5. The method of claim 1 , wherein generating the alert or notification includes performing one or more of:
adjusting a lighting level of the home environment; and
enabling a home device.

6. The method of claim 1 , further comprising, in accordance with the determination that the sound meets the one or more monitoring criteria, enabling a remote occupant of the home environment to perform one or more of: listen to the sound, and listen to the home environment via the one or more microphones.

7. The method of claim 1 , wherein obtaining the determination that the sound meets the one or more monitoring criteria includes obtaining a determination that the sound is an expected sound of the home environment.

8. The method of claim 7 , further comprising obtaining a plurality of previously-recorded home environment sounds; and
wherein obtaining the determination that the sound is not the expected sound comprises comparing the sound to the plurality of previously-recorded home environment sounds.

9. The method of claim 1 , further comprising determining that the sound comprises a voice of an occupant of the home environment.

10. The method of claim 1 , further comprising:
while operating in the second mode, receiving a command to operate in the first mode; and
in accordance with the command, transitioning to operating in the first mode.

11. The method of claim 10 , further comprising:
verifying that the command is from an authorized user; and
transitioning to operating in the first mode in accordance with verification of the command being from the authorized user.

12. The method of claim 1 , wherein the computer system is coupled to one or more additional computer systems within the home environment; and
the method further comprises, in accordance with the determination that the sound meets the one or more monitoring criteria, requesting that at least one additional computer system output the alert or notification.

13. A computer system comprising:
a voice-activated user interface;
one or more microphones;
one or more speakers;
one or more processors coupled to the one or more microphones and the one or more speakers; and
memory coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
while operating in a first mode, receiving a voice command, wherein the first mode is an assistant mode; and
while operating in the first mode, outputting a spoken response to the voice command;
transitioning from operating in the first mode to operating in a second mode, wherein the second mode is a monitoring mode;
while operating in the second mode, detecting a sound;
while operating in the second mode, obtaining a determination that the sound meets one or more monitoring criteria; and
in accordance with a determination that the sound meets the one or more monitoring criteria and while operating in the second mode, generating an alert or notification.

14. The computer system of claim 13 , wherein the alert or notification would not be generated in the first mode.

15. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing system having one or more microphones and one or more speakers, cause the computing system to:
operate in at least two modes comprising a first mode and a second mode, wherein the first mode is an assistant mode and the second mode is a monitoring mode;
while operating in the first mode, receive a voice command; and
while operating in the first mode, output a spoken response to the voice command;
transition from operating in the first mode to operating in the second mode;
while operating in the second mode, detect a sound;
while operating in the second mode, obtain a determination that the sound meets one or more monitoring criteria; and
in accordance with a determination that the sound meets the one or more monitoring criteria and while operating in the second mode, generate an alert or notification.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the alert or notification would not be generated in the first mode.

Assignees

• Google Llc

Inventors

• Horling Bryan

Classifications

• H05B47/197

  Sound control or voice control · CPC title

• H05B47/19

  via wireless transmission · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L25/51

  for comparison or discrimination · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title