Virtual sensor system

Patent metadata

Field	Value
Publication number	US-10436615-B2
Application number	US-201815961537-A
Country	US
Kind code	B2
Filing date	Apr 24, 2018
Priority date	Apr 24, 2017
Publication date	Oct 8, 2019
Grant date	Oct 8, 2019

Abstract

A sensing system includes a sensor assembly that is communicably connected to a computer system, such as a server or a cloud computing system. The sensor assembly includes a plurality of sensors that sense a variety of different physical phenomena. The sensor assembly featurizes the raw sensor data and transmits the featurized data to the computer system. Through machine learning, the computer system then trains a classifier to serve as a virtual sensor for an event that is correlated to the data from one or more sensor streams within the featurized sensor data. The virtual sensor can then subscribe to the relevant sensor feeds from the sensor assembly and monitor for subsequent occurrences of the event. Higher order virtual sensors can receive the outputs from lower order virtual sensors to infer nonbinary details about the environment in which the sensor assemblies are located.

First claim

What is claimed is:

1. A sensing system comprising:
a sensor assembly comprising:
one or more circuit boards;
a control circuit connected to the one or more circuit boards; and
a collection of sensors, at least two of which are heterogeneous, in communication with the control circuit, wherein:
each of the sensors in the collection of sensors is coupled to one or more of the one or more circuit boards such that the each of the sensors in the collection of sensors is configured to sense one or more physical phenomena in an environment of the sensor assembly; and

a back end server system, comprising at least one server, that is in communication with the sensor assembly, wherein:
the control circuit of the sensor assembly is configured to:
extract a plurality of features from raw sensor data collected by the collection of sensors to form featurized data; and
transmit the featurized data to the back end server system; and

the at least one server of the back end server system is configured to:
determine one or more selected sensors of the collection of sensors whose featurized data are correlated with an event occurring in the environment of the sensor assembly;
generate a first order virtual sensor by training a machine learning model to detect the event based on the featurized data from the one or more selected sensors; and
detect the event using the trained first order virtual sensor and featurized data from the one or more selected sensors.

2. The sensing system of claim 1 , wherein the event detected by the first order virtual sensor is not directly sensed by any of the sensors in the collection of sensors of the sensor assembly.

3. The sensing system claim 1 , wherein the at least one server of the back end server system is further configured to generate a second order virtual sensor to detect, based on, at least in part, outputs of the first order virtual sensor, a second order condition in the environment of the sensor assembly.

4. The sensing system of claim 3 , wherein the back end server system is configured to transmit a notification to a remote computer-based system when a particular condition is detected by the second order virtual sensor.

5. The sensing system claim 3 , wherein the second order virtual sensor is a trained machine learning model.

6. The sensing system of claim 1 , wherein the collection of sensors comprise at least one passive sensor selected from the group consisting of an infrared radiation sensor, an ambient light color sensor, an ambient light intensity sensor, a magnetic field sensor, a temperature sensor, an ambient pressure sensor, a humidity sensor, a vibration sensor, an external device communication sensor, a motion sensor, an acoustic sensor, an indoor air quality sensor, a chemical sensor, a vision sensor, and an electromagnetic interference sensor.

7. The sensing system of claim 6 , wherein:
the sensor assembly is in communicating with a user device;
the back end server system is configured to transmit a notification to the sensor assembly when a particular event is detected by the first order virtual sensor; and
the sensor assembly is configured to transmit a notification to the user device in response to receiving the notification from the back end server system that the particular event was detected.

8. The sensing system of claim 1 , wherein the collection of sensors comprise at least one active sensor selected from the group consisting of a sonar sensor, an ultrasonic sensor, a light emitting sensor, a radar based sensor, an acoustic sensor, an infrared camera, an active infrared sensor, an indoor positioning system, an x-ray based sensor, a seismic sensor, and an active sound measurement system.

9. The sensing system of claim 1 , wherein:
the sensing system further comprises an output feedback device selected from the group consisting of a speaker, a light source, and a vibration source;
the back end server system is configured to transmit a notification to the sensor assembly when a particular event is detected by the first order virtual sensor; and
the sensor assembly is configured to transmit a notification to a user via the output feedback device in response to receiving the notification from the back end server system that the particular event was detected.

10. The sensing system of claim 1 , wherein the first order virtual sensor comprises a classifier that is trained to detect the event in the environment of the sensor assembly.

11. The sensing system of claim 10 , wherein the classifier is trained using at least one of supervised learning or unsupervised learning.

12. The sensing system of claim 1 , wherein:
the sensor assembly comprises a housing;
the collection of sensors are connected to the one or more circuit boards; and
the housing houses the one or more circuit boards, the collection of sensors, and the control circuit.

13. The sensing system of claim 1 , wherein the first order virtual sensor produces outputs are selected from the group consisting of a binary output, a non-binary output, and a set of labels.

14. The sensing system of claim 1 , wherein:
the sensor assembly is one of a plurality of sensor assemblies distributed throughout a location, wherein each of the plurality of sensor assemblies is in communication with the back end server system;
each of the plurality of sensor assemblies comprises:
a control circuit; and
a collection of sensors, at least two of which are heterogeneous, connected to the control circuit, wherein each of the sensors in the collection of sensors is configured to sense one or more physical phenomena in a local environment of the sensor assembly, wherein the control circuit of the sensor assembly is configured to:
extract a plurality of features from raw sensor data collected by the collection of sensors to form featurized data; and
transmit the featurized data to the back end server system; and

the first order virtual sensor is trained through machine learning to detect, based on the featurized data transmitted from the plurality of sensor assemblies, the event in the location.

15. The sensing system of claim 1 , wherein the at least one server of the back end server system is configured to transmit to the sensor assembly an instruction for which features to extract for at least one of the sensors in the collection of sensors.

16. The sensing system of claim 1 , wherein the at least one server of the back end server system is configured to identify at least one of the sensors in the collection of sensors that has been activated based on the event in the environment of the sensor assembly.

17. A sensing system comprising:
a sensor assembly comprising:
one or more circuit boards;
a control circuit connected to the one or more circuit boards; and
a collection of sensors, at least two of which are heterogeneous, connected to the control circuit, wherein:
each of the sensors in the collection of sensors is coupled to one or more of the one or more circuit boards such that each of the sensors in the collection of sensors senses one or more physical phenomena in an environment of the sensor assembly that are indicative of events; and
the control circuit is configured to featurize raw sensor data from the collection of sensors to generate featurized data; and

a back end server system, comprising at least one server in communication with the sensor assembly, wherein the at least one server comprises:
a processor; and
a memory storing instructions that, when executed by the processor, cause the at least one server to:
receive the featur

Assignees

• Univ Carnegie Mellon

Inventors

• Agarwal Yuvraj
• Harrison Christopher
• Laput Gierad
• Boovaraghavan Sudershan
• Chen Chen
• Hota Abhijit
• Xiao Bo Robert
• Zhang Yang

Classifications

• G01D3/08

  with provision for safeguarding the apparatus, e.g. against abnormal operation, against breakdown · CPC title

• G06N20/00

  Machine learning · CPC title

• H04L67/12

  specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks · CPC title

• H04L12/2823

  Reporting information sensed by appliance or service execution status of appliance services in a home automation network (device-related reporting H04L43/065; arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom H04Q9/00) · CPC title

• H04W4/38

  for collecting sensor information · CPC title