Systems and methods for monitoring objects and their states by using acoustic signals

Patent metadata

Field	Value
Publication number	US-11200277-B2
Application number	US-201916252509-A
Country	US
Kind code	B2
Filing date	Jan 18, 2019
Priority date	Jan 15, 2016
Publication date	Dec 14, 2021
Grant date	Dec 14, 2021

Abstract

The various implementations disclosed herein pertain to methods and systems for detecting a state of monitored objects in a premises. In one aspect, a method is performed at a computer system that is communicably coupled to sensor devices. The computer system receives and processes information of acoustic signals against a database of stored acoustic signatures characterizing predefined acoustic signals generated by electrically non-powered passive tags in response to physical motion of respective monitored objects associated with the passive tags. It receives a first acoustic signal characterized by a first acoustic signature and generated by a first passive tag. In response to the receiving, based on the first acoustic signature and information in the database, it identifies the first monitored object associated with the first acoustic signature, determines a first state of the first monitored object, and provides a notification of the first state to an occupant of the premises.

First claim

What is claimed is:

1. A method of detecting a state of monitored objects in a premises, comprising:
at a computer system having one or more processors and memory storing instructions for execution by the one or more processors, wherein the computer system is communicatively coupled to one or more sensor devices:
receiving and processing information of acoustic signals against a database of stored acoustic signatures characterizing predefined acoustic signals generated by electrically non-powered passive tags in response to physical motion of respective monitored objects associated with the passive tags;
receiving a first acoustic signal characterized by a first acoustic signature and generated by a first passive tag of the passive tags in response to physical motion of a first monitored object of the respective monitored objects, wherein the first monitored object is associated with the first passive tag; and
in response to the receiving:
based on the first acoustic signature and information in the database:
identifying the first monitored object associated with the first acoustic signature;
determining a first state of a plurality of predefined states of the first monitored object; and
providing a notification of the first state of the first monitored object to an occupant of the premises.

2. The method of claim 1 , wherein the first passive tag comprises a material configured to generate the first acoustic signal in response to physical contact when physically moved in a predetermined manner.

3. The method of claim 1 , wherein at least a subset of the respective monitored objects are entryways of a premises, wherein the entryways include a door or a window.

4. The method of claim 1 , wherein identifying the first monitored object and determining the first state of the first monitored object comprises:
comparing one or more characteristics of the first acoustic signature of the first acoustic signal with one or more characteristics of one or more of the stored acoustic signatures from the database; and
identifying, from the one or more of the stored acoustic signatures from the database, a matching acoustic signature that satisfies a match criterion, wherein the matching acoustic signature is associated with the first monitored object and the first state of the first monitored object.

5. The method of claim 1 , further comprising storing in the database the determined first state of the first monitored object.

6. The method of claim 5 , wherein the storing comprises updating a current state of the first monitored object from a previously stored state to the first state,
wherein the first state and the previously stored state represent distinct states of the first monitored object.

7. The method of claim 6 , further comprising:
detecting a second acoustic signal characterized by the first acoustic signature and generated by the first passive tag;
in response to detecting the second acoustic signal, and based on the first acoustic signature and the information in the database, identifying the first monitored object associated with the first acoustic signature; and
updating the current state of the first monitored object from the first state to the previously stored state in accordance with the second acoustic signal.

8. The method of claim 1 , wherein identifying the first monitored object and/or determining the first state of the first monitored object is further based on one or more additional inputs received from the one or more sensor devices communicably coupled to the computer system.

9. The method of claim 8 , wherein the one or more sensor devices include a camera device, and the one or more additional inputs include video data associated with the detected first acoustic signal.

10. The method of claim 8 , wherein the one or more additional inputs include current presence detection data associated with the detected first acoustic signal.

11. The method of claim 8 , wherein the one or more sensor devices include a thermostat device, and the one or more additional inputs include temperature data associated with the detected first acoustic signal.

12. The method of claim 8 , wherein the one or more sensor devices include an acoustic sensor, and the one or more additional inputs include additional acoustic data associated with the detected first acoustic signal.

13. The method of claim 8 , wherein the one or more additional inputs include a user input confirming the first state of the first monitored object.

14. A non-transitory computer-readable storage medium storing one or more programs for execution by one or more processors of a computer system, the one or more programs including instructions for:
receiving and processing information of acoustic signals against a database of stored acoustic signatures characterizing predefined acoustic signals generated by electrically non-powered passive tags in response to physical motion of respective monitored objects associated with the passive tags;
receiving a first acoustic signal characterized by a first acoustic signature and generated by a first passive tag of the passive tags in response to physical motion of a first monitored object of the respective monitored objects, wherein the first monitored object is associated with the first passive tag; and
in response to the receiving:
based on the first acoustic signature and information in the database:
identifying the first monitored object associated with the first acoustic signature;
determining a first state of a plurality of predefined states of the first monitored object; and
providing a notification of the first state of the first monitored object to an occupant of the premises.

15. The non-transitory computer readable storage medium of claim 14 , wherein:
the one or more sensor devices are located within a premises, and the computer system is a remote server located outside of the premises; and
receiving the first acoustic signal comprises receiving, at the computer system and from the one or more sensor devices, the first acoustic signal.

16. The non-transitory computer readable storage medium of claim 14 , wherein acoustic signatures, which include the stored acoustic signatures in the database and the first acoustic signature of the first acoustic signal, are defined by one or more characteristics of acoustic signatures, the characteristics including at least one of:
an acoustic signature length;
one or more amplitude values; and
one or more spectral coefficients for one or more respective frequencies.

17. The non-transitory computer readable storage medium of claim 16 , wherein at least some of the stored acoustic signatures are defined by one or more of the characteristics excluding amplitude values.

18. The non-transitory computer readable storage medium of claim 16 , wherein each or any combination of the one or more characteristics respectively defining the stored acoustic signatures is associated with: a respective object, a type of object, a state or change of state for a respective object, a state or change of state for a respective type of object, and/or a location.

19. The non-transitory computer readable storage medium of claim 14 , further comprising instructions for, prior to processing received information of acoustic signals against the database:
creating one or more entries of the database, including:
detecting the predefined acoustic signals, each of the predefined acoustic signals characterized by a respective stored acoustic signature, and generated by a respective passiv

Assignees

• Google Llc

Inventors

• Tannenbaum Harry
• Irvine Benjamin
• Sayadi Shayan
• Singer James Vanhook

Classifications

• G06F16/686 Primary

  using information manually generated, e.g. tags, keywords, comments, title or artist information, time, location or usage information, user ratings · CPC title

• H04L12/2825 Primary

  Reporting to a device located outside the home and the home network (access arrangements H04L12/2856; for remote control or remote monitoring of applications H04L67/025; telephonic communication systems adapted for combination with telemetering systems H04M11/002) · CPC title

• G06F16/61

  Indexing; Data structures therefor; Storage structures · CPC title