Informative image data generation using audio/video recording and communication devices

Patent metadata

Field	Value
Publication number	US-10769914-B2
Application number	US-201816003035-A
Country	US
Kind code	B2
Filing date	Jun 7, 2018
Priority date	Jun 7, 2017
Publication date	Sep 8, 2020
Grant date	Sep 8, 2020

Abstract

Some embodiments provide for receiving image data representative of a field of view of a camera of an audio/video (A/V) recording and communication device during a period time and sensor data generated by a sensor or a hub device at a time. A time-based association between the A/V recording and communication device and the sensor may be determined. Based at least in part on the time and the period of time, it may be determined that the time is within the period of time and that the time being within the period of time satisfies the time-based association. The image data and the sensor data may be correlated to create updated data, and the updated data may be included within a generated message. The message may then be transmitted to a client device associated with the A/V recording and communication device.

First claim

What is claimed is:

1. A method comprising:
receiving, from an A/V recording and communication device, image data generated during a period of time by a camera of the A/V recording and communication device, the image data representative of images of a field of view of the camera;
receiving, from at least one of a sensor or a hub device, sensor data generated by the sensor in response to a sensor trigger at a time;
determining a time-based association between the A/V recording and communication device and the sensor;
determining an object-based association between the sensor data and an object present within the field of view of the camera;
determining that the time is within the period of time;
correlating the sensor data with the image data to generate updated data, wherein the updated data associates the sensor data with the object present within the field of view of the camera during the period of time;
generating a message including the updated data; and
transmitting the message to a client device associated with the A/V recording a communication device, the updated data configured to cause the images of the field of view of the camera represented by the image data to be displayed in addition to a representation of the sensor data positioned proximate to the object present within the field of view of the camera.

2. The method of claim 1 , wherein the sensor includes at least one of:
a door sensor, the sensor trigger is a door opening/closing, and the sensor data is representative of the door opening or closing;
a window sensor, the sensor trigger is a window opening/closing, and the sensor data is representative of the window opening or closing;
a temperature sensor, the sensor trigger is a temperature over a threshold, and the sensor data is representative of the temperature;
a carbon monoxide sensor, the sensor trigger is a carbon monoxide level over a threshold, and the sensor data is representative of the carbon monoxide level;
a smoke alarm, the sensor trigger is a threshold amount of smoke detected, and the sensor data is representative of the amount of smoke detected;
a light sensor, the sensor trigger is an amount of ambient light being above or below a threshold, and the sensor data is representative of the amount of ambient light; or
a motion sensor, the sensor trigger is a value determined from a signal generated by the motion sensor being greater than a threshold, and the sensor data is representative of the value.

3. The method of claim 1 , wherein the image data is received in response the camera or a motion sensor of the A/V recording and communication device detecting the object.

4. The method of claim 1 , wherein:
the period of time is determined based at least in part on first clock data associated with the image data;
the time is determined based at least in part on second clock data associated with the sensor data; and
the determining that the time is within the period of time is based at least in part on the first clock data and the second clock data.

5. The method of claim 1 , wherein the correlating the image data and the sensor data to create the updated data comprises:
matching the time associated with the sensor data with the time within the period of time associated with the image data;
correlating the sensor data with at least a portion of the image data corresponding to the time within the period of time; and
creating the updated data based at least in part on the correlating the sensor data with at least the portion of the image data corresponding to the time within the period of time.

6. The method of claim 1 , wherein:
the representation is a first representation;
the images are first images associated with the period of time including and after the time; and
prior to the display of the first images represented by the image data and the representation of the sensor data, the updated data is configured to cause second images of the field of view of the camera represented by the image data associated with the period of time prior to the time to be displayed in addition to a second representation of other sensor data generated by at least one of the sensor or another sensor.

7. The method of claim 1 , wherein the representation of the sensor data is configured to be overlaid onto the images represented by the image data.

8. The method of claim 1 , wherein:
the message is configured to be displayed as a push-notification on the client device; and
the push-notification is configured to include another representation of the sensor data and at least one of the images represented by the image data.

9. A method comprising:
receiving, from an A/V recording and communication device and during a first portion of a period of time, first image data generated by a camera of the A/V recording and communication device, the first image data being representative of first images of a field of view of the camera;
receiving, from a first sensor, first sensor data generated at a first time within the first portion of the period of time;
generating a first message including the first image data and the first sensor data, the first message configured to cause display of first images represented by the first image data in addition to a first representation of the first sensor data;
transmitting the first message to a client device associated with the A/V recording and communication device;
receiving, from the A/V recording and communication device and during a second portion of the period of time after the first portion, second image data generated by the camera of the A/V recording and communication device, the second image data being representative of second images of the field of view of the camera;
receiving, from one of the first sensor or a second sensor, second sensor data generated based at least in part on a sensor trigger and at a second time within the second portion of the period of time;
determining an object-based association between the second sensor data and an object present within the field of view of the camera;
generating a second message including the second image data and the second sensor data, the second message configured to cause the second images of the field of view of the camera represented by the second image data to be displayed in addition to a second representation of the second sensor data positioned proximate to the object present within the field of view of the camera; and
transmitting the second message to the client device.

10. The method of claim 9 , wherein at least one of the sensor or the second sensor includes:
a door sensor, the sensor trigger is a door opening/closing, and the second sensor data is representative of the door opening or closing;
a window sensor, the sensor trigger is a window opening/closing, and the second sensor data is representative of the window opening or closing;
a temperature sensor, the sensor trigger is a temperature over a threshold, and the second sensor data is representative of the temperature;
a carbon monoxide sensor, the sensor trigger is a carbon monoxide level over a threshold, and the second sensor data is representative of the carbon monoxide level;
a smoke alarm, the sensor trigger is a threshold amount of smoke detected, and the second sensor data is representative of the amount of smoke detected;
a light sensor, the sensor trigger is an amount of ambient light being above or below a threshold, and the second sensor data is representative of the amount of ambient light; or
a motion sensor, the sensor trigger is a value determined from a signal generated by the motion sensor being greater than a threshold, and the second sensor data is representative of the value.

Assignees

• Amazon Tech Inc

Inventors

• Siminoff James

Classifications

• G08B13/19691 Primary

  Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound · CPC title

• G08B13/189

  using passive radiation detection systems · CPC title

• G08B13/19669

  Event triggers storage or change of storage policy · CPC title

• G08B13/19684

  Portable terminal, e.g. mobile phone, used for viewing video remotely · CPC title

• G08B13/19671

  Addition of non-video data, i.e. metadata, to video stream · CPC title