Systems, methods, and devices for activity monitoring via a home assistant

Patent metadata

Field	Value
Publication number	US-10650647-B2
Application number	US-201916536212-A
Country	US
Kind code	B2
Filing date	Aug 8, 2019
Priority date	May 12, 2017
Publication date	May 12, 2020
Grant date	May 12, 2020

Abstract

The various implementations described herein include methods, devices, and systems for monitoring activity in a home environment. In one aspect, a method performed at a voice-assistant device includes: while operating in an assistant mode, receiving an occupant command to operate in a monitoring mode; in response to the command, transitioning to operating in the monitoring mode; while operating in the monitoring mode, detecting a sound; obtaining a determination as to whether the sound meets one or more monitoring criteria; and in accordance with a determination that the sound meets the one or more monitoring criteria: obtaining a classification of the sound; and generating an alert or notification in accordance with the classification of the sound.

First claim

What is claimed is:

1. A method for monitoring activity in a home environment, comprising:
at a computer system having a voice-activated user interface, one or more microphones, one or more speakers, one or more processors, and memory, wherein the computer system has at least two distinct operating modes, including an assistant mode and a monitoring mode:
while operating in the assistant mode, receiving an occupant command to operate in the monitoring mode;
in response to the occupant command, transitioning from operating in the assistant mode to operating in the monitoring mode;
while operating in the monitoring mode, detecting a sound;
obtaining a determination as to whether the sound meets one or more monitoring criteria; and
in accordance with a determination that the sound meets the one or more monitoring criteria:
obtaining a classification of the sound; and
generating an alert or notification in accordance with the classification of the sound.

2. The method of claim 1 , wherein generating the alert or notification includes emitting a first response of a plurality of responses via the one or more speakers, the plurality of responses includes one or more of:
a simulated occupant response;
a simulated animal response;
a pre-recorded occupant response;
a request for identification; and
a simulated human response.

3. The method of claim 1 , wherein generating the alert or notification includes performing one or more of:
adjusting a lighting level of the home environment;
adjusting a user interface of the computer system to indicate that the sound meeting the one or more monitoring criteria was detected;
storing the sound;
enabling a home device; and
sending an alert.

4. The method of claim 1 , further comprising, in accordance with the determination that the sound does not meet the one or more monitoring criteria, forgoing obtaining the classification of the sound.

5. The method of claim 1 , further comprising, based on the sound having a second sound classification, forgoing generating an alert or notification.

6. The method of claim 1 , further comprising, in accordance with the determination that the sound meets the one or more monitoring criteria, enabling a remote occupant of the home environment to perform one or more of: listen to the sound, listen to the home environment via the one or more microphones.

7. The method of claim 1 , wherein obtaining the determination as to whether the sound meets the one or more monitoring criteria includes obtaining a determination as to whether the sound is an expected sound of the home environment.

8. The method of claim 7 , further comprising obtaining a plurality of previously-recorded home environment sounds; and
wherein obtaining the determination that the sound is not an expected sound of the home environment comprises comparing the sound to the plurality of previously-recorded home environment sounds.

9. The method of claim 1 , wherein obtaining the classification of the sound comprises obtaining a determination as to whether the sound comprises a voice of an occupant of the home environment.

10. The method of claim 1 , wherein obtaining the classification of the sound includes obtaining a determination as to whether an occupant of the home environment is expected to currently be within the home environment.

11. The method of claim 10 , wherein generating the alert or notification includes emitting a first response of a plurality of responses via the one or more speakers; and the method further comprises, in accordance with a determination that the occupant of the home environment is expected to currently be within the home environment, emitting a second response via the one or more speakers, the second response distinct from the first response.

12. The method of claim 1 , further comprising:
receiving a second occupant command to operate the computer system in a learning mode;
in response to the second occupant command, transitioning to operating in the learning mode;
while operating in the learning mode, analyzing sounds of the home environment to identify a plurality of expected sounds within the home environment.

13. The method of claim 1 , further comprising:
while operating in the monitoring mode, receiving a second occupant command to operate in the assistant mode; and
in accordance with the second occupant command, transitioning to operating in the assistant mode.

14. The method of claim 13 , further comprising:
verifying that the second occupant command comprises a command from an authorized user; and
transitioning to operating in the assistant mode in accordance with verification of the second occupant command.

15. The method of claim 1 , wherein the computer system is coupled to one or more additional computer systems within the home environment; and
the method further comprises, in accordance with the determination that the sound meets the one or more monitoring criteria, requesting that at least one of the one or more additional computer systems generate an alert or notification.

16. The method of claim 1 , wherein the computer system is coupled to one or more additional computer system within the home environment; and
the method further comprises:
receiving notification of a second sound meeting one or more monitoring criteria from at least one of the one or more additional computer systems; and
in response to receiving notification of the second sound, emitting a response via the one or more speakers.

17. The method of claim 1 , further comprising, while in the monitoring mode, mimicking occupant activity within the home environment.

18. The method of claim 1 , wherein generating the alert or notification includes emitting a first response of a plurality of responses via the one or more speakers, and the method further comprises:
while in the monitoring mode, anticipating a particular sound within the home environment at a particular time; and
in accordance with a determination that the particular sound did not occur at the particular time, emitting a second response via the one or more speakers.

19. A computer system comprising:
a voice-activated user interface;
one or more microphones;
one or more speakers;
one or more processors coupled to the one or more microphones and the one or more speakers; and
memory coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
operating in at least two distinct modes, including an assistant mode and a monitoring mode;
while operating in the assistant mode, receiving an occupant command to operate in the monitoring mode;
in response to the occupant command, transitioning from operating in the assistant mode to operating in the monitoring mode;
while operating in the monitoring mode, detecting a sound;
obtaining a determination as to whether the sound meets one or more monitoring criteria; and
in accordance with a determination that the sound meets the one or more monitoring criteria:
obtaining a classification of the sound; and
generating an alert or notification in accordance with the classification of the sound.

20. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing system having one or more microphones and one or more speakers, cause the system to:
operate in at least two distinct modes, inclu

Assignees

• Google Llc

Inventors

• Horling Bryan

Classifications

• G08B15/002 Primary

  with occupancy simulation · CPC title

• G05B15/02

  electric · CPC title

• G08B25/008

  Alarm setting and unsetting, i.e. arming or disarming of the security system · CPC title

• G08B13/1672 Primary

  using sonic detecting means, e.g. a microphone operating in the audio frequency range · CPC title

• G05B2219/2642

  Domotique, domestic, home control, automation, smart house · CPC title