Using implicit event ground truth for video cameras

Patent metadata

Field	Value
Publication number	US-12417636-B2
Application number	US-202318235918-A
Country	US
Kind code	B2
Filing date	Aug 21, 2023
Priority date	Aug 25, 2022
Publication date	Sep 16, 2025
Grant date	Sep 16, 2025

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for object detection. One of the methods includes determining, using first sensor data, a detection result on whether to trigger an event alerting a presence of an object in a target area by executing one or more models; determining, using second sensor data, a ground truth for the event that indicates whether an object is present in the target area; determining a difference value by comparing the detection result and the ground truth; adjusting at least one parameter of the one or more models in response to determining that the difference value does not satisfy the one or more threshold criteria; and determining a new detection result on whether to trigger a second event by executing the one or more models with adjusted parameters using new first sensor data.

First claim

The invention claimed is:

1. A computer-implemented method comprising:
receiving, by one or more computing devices, first sensor data collected by one or more first sensors of a monitoring system;
determining, by the one or more computing devices and using the first sensor data, a detection result on whether to trigger an event alerting a presence of an object in a target area by executing one or more models of an object detection process;
after receiving the first sensor data,
receiving, by the one or more computing devices, second sensor data from one or more second sensors of the monitoring system and that indicates an action performed by an object;
determining, by the one or more computing devices and using the second sensor data, a ground truth for the event that indicates whether an object is present in the target area and that the event actually occurred;

determining, by the one or more computing devices, a difference value representing a degree of accuracy of the one or more models for the event by comparing the detection result and the ground truth;
determining, by the one or more computing devices, whether the difference value satisfies one or more threshold criteria;
adjusting, by the one or more computing devices, at least one parameter of the one or more models in response to determining that the difference value does not satisfy the one or more threshold criteria; and
determining, by the one or more computing devices, a new detection result on whether to trigger a second event by executing the one or more models with adjusted parameters using new first sensor data.

2. The computer-implemented method of claim 1 , comprising:
in response to determining to trigger the event using the first sensor data, triggering the event before at least one of adjusting the at least one parameter of the one or more models, determining the ground truth for the event using the second sensor data, determining the difference value, or determining whether the difference value satisfies the one or more threshold criteria.

3. The computer-implemented method of claim 2 , comprising:
determining whether to adjust at least one of the models using a first timestamp of the triggering of the event and a second timestamp of the ground truth; and
adjusting at least another parameter of the one or more models using the first timestamp of the triggering of the event and the second timestamp of ground truth.

4. The computer-implemented method of claim 3 , wherein:
adjusting at least the other parameter of the one or more models using a difference between the first timestamp of the triggering of the event and the second timestamp of ground truth.

5. The computer-implemented method of claim 1 , comprising:
determining, by one or more computing devices, a second difference value representing a degree of accuracy of the one or more models by comparing i) a detection result for a second event determined using third sensor data captured by a sensor of the monitoring system and ii) a ground truth for the second event determined using fourth sensor data captured by another sensor of the monitoring system;
determining, by the one or more computing devices, whether the second difference value satisfies the one or more threshold criteria; and
determining to skip adjusting the at least one parameter of the one or more models in response to determining that the second difference value satisfies the one or more threshold criteria.

6. The computer-implemented method of claim 1 , wherein:
the one or more first sensors of the monitoring system comprise at least one of a camera and a motion detector, and
the one or more second sensors comprise at least one of a camera, a motion detector, a doormat, a button, an audio sensor, a glass break sensor, a pressure sensor, a distance sensor, a door open sensor, a doorbell, or a passive infrared (PIR) sensor.

7. The computer-implemented method of claim 1 , wherein determining the detection result comprises:
comparing the first sensor data with an object data to determine whether the first sensor data satisfies a similarity threshold for the object; and
in response to determining that the first sensor data satisfies the similarity threshold, determining that the object is present in the target area and determining to trigger the event.

8. The computer-implemented method of claim 1 , wherein determining the detection result comprises:
comparing the first sensor data with background image data to determine whether a difference satisfies a threshold; and
in response to determining that the difference satisfies the threshold, determining that an object is present in the target area and determining to trigger the event.

9. The computer-implemented method of claim 1 , wherein determining whether the difference value satisfies the one or more threshold criteria comprises:
determining whether a first timestamp for the event satisfies a timing threshold for a second timestamp of the ground truth, the timing threshold representing an acceptable range of time for trigger the event.

10. The computer-implemented method of claim 1 , wherein:
determining, using the first sensor data, the detection result comprises performing two or more actions to generate the detection result; and
adjusting at least the one parameter of the one or more models comprises adjusting at least the one parameter of the one or more models using a first timestamp for a particular action from the two or more actions and a second timestamp for the ground truth.

11. The computer-implemented method of claim 10 , wherein:
determining whether the difference value satisfies one or more threshold criteria comprises determining whether the first timestamp of the particular action from the two or more actions does not satisfy the one or more threshold criteria compared to the second timestamp of the ground truth; and
adjusting at least the one parameter of the one or more models comprises adjusting one or more parameters of a model using data for the particular action.

12. The computer-implemented method of claim 11 , wherein adjusting the one or more parameters of the model comprises:
selecting a model that performed the particular action; and
adjusting the one or more parameters of the model that performed the particular action.

13. One or more non-transitory computer storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform:
receiving first sensor data collected by one or more first sensors of a monitoring system;
determining, using the first sensor data, a detection result on whether to trigger an event alerting a presence of an object in a target area by executing one or more models of an object detection process;
after receiving the first sensor data,
receiving second sensor data from one or more second sensors of the monitoring system and that indicates an action performed by an object;
determining, using the second sensor data, a ground truth for the event that indicates whether an object is present in the target area and that the event actually occurred;
determining a difference value representing a degree of accuracy of the one or more models for the event by comparing the detection result and the ground truth;
determining whether the difference value satisfies one or more threshold criteria;
adjusting at least one parameter of the one or more models in response to determining that the difference value does not satisfy the one or more threshold criteria; and
determining a new detection result on whether to trigger a second event by executing the one or more models with a

Assignees

• Objectvideo Labs Llc

Inventors

• Ramanathan Narayanan
• Beach Allison
• Qian Gang
• Park Kyoung-Jin
• Madden Donald Gerard
• Shayne Ethan

Classifications

• G08B29/24

  Self-calibration, e.g. compensating for environmental drift or ageing of components · CPC title

• G08B29/188

  Data fusion; cooperative systems, e.g. voting among different detectors · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06V2201/07

  Target detection · CPC title