Systems, methods, and devices for activity monitoring via a home assistant

Patent metadata

Field	Value
Publication number	US-10380852-B2
Application number	US-201815895910-A
Country	US
Kind code	B2
Filing date	Feb 13, 2018
Priority date	May 12, 2017
Publication date	Aug 13, 2019
Grant date	Aug 13, 2019

Abstract

The various implementations described herein include methods, devices, and systems for monitoring activity in a home environment. In one aspect, a method performed at a voice-assistant device includes: (1) while operating in an assistant mode, receiving an occupant voice command to operate in a monitoring mode; (2) in response to the command, transitioning to operating in the monitoring mode; (3) obtaining one or more monitoring criteria; (4) while operating in the monitoring mode, detecting a sound; (5) obtaining a determination as to whether the sound meets the one or more monitoring criteria; and (6) in accordance with a determination that the sound meets the one or more monitoring criteria: (a) obtaining a classification of the sound; and (b) based on sound having a first sound classification, emitting a first simulated occupant response of a plurality of simulated occupant responses via the one or more speakers.

First claim

What is claimed is:

1. A method for monitoring activity in a home environment, comprising:
at a computer system having a voice-activated user interface, one or more microphones, one or more speakers, one or more processors, and memory, wherein the computer system has at least two distinct operating modes, including an assistant mode and a monitoring mode:
while operating in the assistant mode, receiving an occupant voice command to operate in the monitoring mode;
in response to the occupant command, transitioning from operating in the assistant mode to operating in the monitoring mode;
obtaining one or more monitoring criteria;
while operating in the monitoring mode, detecting a sound;
obtaining a determination as to whether the sound meets the one or more monitoring criteria; and
in accordance with a determination that the sound meets the one or more monitoring criteria:
obtaining a classification of the sound; and
based on sound having a first sound classification, emitting a first simulated occupant response of a plurality of simulated occupant responses via the one or more speakers.

2. The method of claim 1 , wherein the plurality of simulated occupant responses includes one or more of:
a simulated animal response;
a pre-recorded occupant response;
a request for identification; and
a simulated human response.

3. The method of claim 1 , further comprising, in accordance with the determination that the sound meets the one or more monitoring criteria, performing one or more of:
adjusting a lighting level of the home environment;
adjusting a user interface of the computer system to indicate that the sound meeting the one or more predefined criteria was detected;
storing the sound;
enabling a home device; and
sending an alert.

4. The method of claim 1 , further comprising, in accordance with the determination that the sound does not meet the one or more monitoring criteria, forgoing obtaining the classification of the sound.

5. The method of claim 1 , further comprising, based on the sound having a second sound classification, forgoing emitting a simulated occupant response via the one or more speakers.

6. The method of claim 1 , further comprising, in accordance with the determination that the sound meets the one or more monitoring criteria, enabling a remote occupant of the home environment to perform one or more of: listen to the sound, listen to the home environment via the one or more microphones.

7. The method of claim 1 , wherein obtaining the determination as to whether the sound meets the one or more predefined criteria includes obtaining a determination as to whether the sound is an expected sound of the home environment.

8. The method of claim 7 , further comprising obtaining a plurality of previously-recorded home environment sounds; and
wherein obtaining the determination that the sound is not an expected sound of the home environment comprises comparing the sound to the plurality of previously-recorded home environment sounds.

9. The method of claim 1 , wherein obtaining the classification of the sound comprises obtaining a determination as to whether the sound comprises a voice of an occupant of the home environment.

10. The method of claim 1 , wherein obtaining the classification of the sound includes obtaining a determination as to whether an occupant of the home environment is expected to currently be within the home environment.

11. The method of claim 10 , further comprising, in accordance with a determination that the occupant of the home environment is expected to currently be within the home environment, emitting a second simulated occupant response via the one or more speakers, the second simulated occupant response distinct from the simulated occupant response.

12. The method of claim 1 , further comprising:
receiving a second occupant command to operate the computer system in a learning mode;
in response to the second occupant command, transitioning to operating in the learning mode;
while operating in the learning mode, analyzing sounds of the home environment to identify a plurality of expected sounds within the home environment.

13. The method of claim 1 , further comprising:
while operating in the monitoring mode, receiving a second occupant command to operate in the assistant mode; and
in accordance with the second occupant command, transitioning to operating in the assistant mode.

14. The method of claim 13 , further comprising:
verifying that the second occupant command comprises a command from an authorized user; and
transitioning to operating in the assistant mode in accordance with verification of the second occupant command.

15. The method of claim 1 , wherein the computer system is coupled to one or more additional computer systems within the home environment; and
the method further comprises, in accordance with the determination that the sound meets the one or more monitoring criteria, requesting that at least one of the one or more additional computer systems simulates an occupant response to the sound.

16. The method of claim 1 , wherein the computer system is coupled to one or more additional computer system within the home environment; and
the method further comprises:
receiving notification of a second sound meeting one or more predefined criteria from at least one of the one or more additional computer systems; and
in response to receiving notification of the second sound, emitting a second simulated occupant response via the one or more speakers.

17. The method of claim 1 , further comprising, while in the monitoring mode, mimicking occupant activity within the home environment.

18. The method of claim 1 , further comprising:
while in the monitoring mode, anticipating a particular sound within the home environment at a particular time; and
in accordance with a determination that the particular sound did not occur at the particular time, emitting a second simulated occupant response via the one or more speakers.

19. A computer system comprising:
a voice-activated user interface;
one or more microphones;
one or more speakers;
one or more processors coupled to the one or more microphones and the one or more speakers; and
memory coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
operating in at least two distinct modes, including an assistant mode and a monitoring mode;
while operating in the assistant mode, receiving an occupant voice command to operate in the monitoring mode;
in response to the occupant command, transitioning from operating in the assistant mode to operating in the monitoring mode;
obtaining one or more monitoring criteria;
while operating in the monitoring mode, detecting a sound;
obtaining a determination as to whether the sound meets the one or more monitoring criteria; and
in accordance with a determination that the sound meets the one or more monitoring criteria:
obtaining a classification of the sound; and
based on sound having a first sound classification, emitting a first simulated occupant response of a plurality of simulated occupant responses via the one or more speakers.

20. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing system having one or more microphones and one or more speakers, cause the system to:
operate in a

Assignees

• Google Llc

Inventors

• Horling Bryan

Classifications

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• H05B47/125

  by using cameras · CPC title

• H05B47/12

  by detecting audible sound · CPC title

• H05B47/19

  via wireless transmission · CPC title

• G08B25/008

  Alarm setting and unsetting, i.e. arming or disarming of the security system · CPC title