Systems and methods for monitoring objects and their states by using acoustic signals

Patent metadata

Field	Value
Publication number	US-11734345-B2
Application number	US-202117523647-A
Country	US
Kind code	B2
Filing date	Nov 10, 2021
Priority date	Jan 15, 2016
Publication date	Aug 22, 2023
Grant date	Aug 22, 2023

Abstract

The various implementations disclosed herein pertain to methods and systems for detecting a state of monitored objects in a premises. In one aspect, a method is performed at a computer system that is communicably coupled to sensor devices. The computer system receives and processes information of acoustic signals against a database of stored acoustic signatures characterizing predefined acoustic signals generated by electrically non-powered passive tags in response to physical motion of respective monitored objects associated with the passive tags. It receives a first acoustic signal characterized by a first acoustic signature and generated by a first passive tag. In response to the receiving, based on the first acoustic signature and information in the database, it identifies the first monitored object associated with the first acoustic signature, determines a first state of the first monitored object, and provides a notification of the first state to an occupant of the premises.

First claim

What is claimed is:

1. A method of detecting a state of monitored objects in a premises, comprising:
at a computer system having one or more processors and memory storing instructions for execution by the one or more processors, wherein the computer system is communicatively coupled to one or more sensor devices:
receiving and processing information of acoustic signals against a database of stored acoustic signatures characterizing predefined acoustic signals generated by electrically non-powered passive tags in response to physical motion of respective monitored objects associated with the passive tags;
receiving a first acoustic signal characterized by a first acoustic signature and generated by a first passive tag of the passive tags in response to physical motion of a first monitored object of the respective monitored objects, wherein the first monitored object is associated with the first passive tag; and
in response to the receiving:
based on the first acoustic signature and information in the database:
identifying the first monitored object associated with the first acoustic signature;
determining a first state of a plurality of predefined states of the first monitored object; and
providing at least one of:
 an indication of the first state to an application on a device associated with the first monitored object;
 the indication of the first state to a smart home server system associated with the first monitored; or
 a command to an additional monitored object of the respective monitored objects to enable the additional monitored object to perform an action in response to the first state.

2. The method of claim 1 , wherein the additional monitored object is a video surveillance camera and the command to enable the additional monitored object to perform the action in response to the first state causes the video surveillance camera to begin capturing video.

3. The method of claim 1 , wherein each predefined state of the plurality of predefined states is based on one or more previous acoustic signals having respective acoustic signatures.

4. The method of claim 3 , wherein the one or more previous acoustic signals having respective acoustic signatures comprises:
a collection of respective acoustic signals collected by at least two of the one or more sensor devices; or
a collection of two or more acoustic signals collected by a single sensor device of the one or more sensor devices.

5. The method of claim 1 , wherein:
the passive tags comprise an adhesive material with a corresponding degree of adhesiveness; and
the adhesive material produces an acoustic signal having a particular acoustic signature in response to physical contact.

6. The method of claim 1 , further comprising storing in the database the determined first state of the first monitored object.

7. The method of claim 6 , wherein:
the storing comprises updating a current state of the first monitored object from a previously stored state to the first state; and
the first state and the previously stored state represent distinct states of the first monitored object.

8. The method of claim 7 , further comprising:
detecting a second acoustic signal having a second acoustic signature generated by the first passive tag;
in response to detecting the second acoustic signal, and based on the second acoustic signature and the information in the database, identifying the first monitored object associated with the second acoustic signature; and
requesting, to the device associated with the first monitored object, authorization to associate the second acoustic signature with the first monitored object and a respective state corresponding to the second acoustic signature.

9. The method of claim 8 , further comprising:
in response to receiving, from the device associated with the first monitored object, the authorization to associate the second acoustic signature with the first monitored object and the respective state corresponding to the second acoustic signature, storing in the database the respective state corresponding to the second acoustic signature of the first monitored object.

10. The method of claim 9 , wherein the authorization includes a user input confirming the respective state corresponding to second acoustic signature of the first monitored object.

11. A non-transitory computer-readable storage medium storing one or more programs for execution by one or more processors of a computer system, the one or more programs comprising instructions for:
receiving and processing information of acoustic signals against a database of stored acoustic signatures characterizing predefined acoustic signals generated by electrically non-powered passive tags in response to physical motion of respective monitored objects associated with the passive tags;
receiving a first acoustic signal characterized by a first acoustic signature and generated by a first passive tag of the passive tags in response to physical motion of a first monitored object of the respective monitored objects, wherein the first monitored object is associated with the first passive tag; and
in response to the receiving:
based on the first acoustic signature and information in the database:
identifying the first monitored object associated with the first acoustic signature;
determining a first state of a plurality of predefined states of the first monitored object; and
providing at least one of:
 an indication of the first state to an application on a device associated with the first monitored object;
 the indication of the first state to a smart home server system associated with the first monitored; or
 a command to an additional monitored object of the respective monitored objects to enable the additional monitored object to perform an action in response to the first state.

12. The non-transitory computer readable storage medium of claim 11 , wherein the additional monitored object is a video surveillance camera and the command to enable the additional monitored object to perform the action in response to the first state causes the video surveillance camera to begin capturing video.

13. The non-transitory computer readable storage medium of claim 11 , wherein each of the plurality of predefined states are based on one or more previous acoustic signals having respective acoustic signatures.

14. The non-transitory computer readable storage medium of claim 11 , wherein the one or more programs further comprise instructions for storing in the database the determined first state of the first monitored object.

15. The non-transitory computer readable storage medium of claim 14 , wherein the storing comprises updating a current state of the first monitored object from a previously stored state to the first state,
wherein the first state and the previously stored state represent distinct states of the first monitored object.

16. The non-transitory computer readable storage medium of claim 15 , wherein the one or more programs further comprise instructions for:
detecting a second acoustic signal having a second acoustic signature generated by the first passive tag;
in response to detecting the second acoustic signal, and based on the second acoustic signature and the information in the database, identifying the first monitored object associated with the second acoustic signature; and
requesting, to the device associated with the first monitored object, authorization to associate the second acoustic signature with the first monitored object and a respective state corresponding to the second acoustic signature.

17. T

Assignees

• Google Llc

Inventors

• Tannenbaum Harry
• Irvine Benjamin
• Sayadi Shayan
• Singer James Van Hook

Classifications

• G06F16/686 Primary

  using information manually generated, e.g. tags, keywords, comments, title or artist information, time, location or usage information, user ratings · CPC title

• H04L12/2825 Primary

  Reporting to a device located outside the home and the home network (access arrangements H04L12/2856; for remote control or remote monitoring of applications H04L67/025; telephonic communication systems adapted for combination with telemetering systems H04M11/002) · CPC title

• G06F16/61

  Indexing; Data structures therefor; Storage structures · CPC title