Monitoring presence or absence of an object using local region matching

Patent metadata

Field	Value
Publication number	US-12299989-B2
Application number	US-202217721667-A
Country	US
Kind code	B2
Filing date	Apr 15, 2022
Priority date	Apr 15, 2021
Publication date	May 13, 2025
Grant date	May 13, 2025

Abstract

Methods and systems, including computer-readable media, are described for monitoring presence or absence of an object at a property using local region matching. A system generates images while monitoring an area of the property and, based on the images, detects an object in a region of interest in the area. For each of the images: the system iteratively computes interest points for the object using photometric augmentation applied to the image before each iteration of computing the interest points. A digital representation of the region and the object is generated based on interest points that repeat across the images after each application of the photometric augmentation. Based on the digital representation, a set of anchor points are determined from the interest points that repeat across images. Using the set of anchor points, the system detects an absence or a continued presence of the object in the area of the property.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
maintaining a plurality of images that were generated while monitoring an area of a property;
detecting, using at least some of the plurality of images, an object in a region of interest in the area;
for at least some images from the plurality of images:
iteratively computing interest points for the object based on photometric augmentation applied to the image before each iteration of computing the interest points, wherein the photometric augmentation comprises:
performing a Scale Invariant Feature Transform (SIFT) operation on the image;

generating a digital representation of the region and the object based on interest points that repeat across the plurality of images after each application of the photometric augmentation;
determining, based on the digital representation, a set of anchor points from the interest points that repeat across the plurality of images; and
using the set of anchor points, detecting an absence of the object in the area of the property or a continued presence of the object in the area of the property.

2. The method of claim 1 , wherein determining a set of anchor points comprises:
determining a respective set of descriptors for each anchor point in the set of anchor points, wherein each respective set of descriptors represents a local region around an interest point from which the corresponding anchor point was determined.

3. The method of claim 2 , wherein determining a set of anchor points from the interest points that repeat across the plurality of images comprises:
computing a respective probability value for each of the interest points that repeat across the plurality of images;
aggregating interest points having a respective probability value that exceeds a threshold probability value; and
determining the set of anchor points based on the aggregated interest points.

4. The method of claim 2 , further comprising:
determining a first bag of descriptors representing a first local region around a first anchor point for a first image of the area; and
determining a second bag of descriptors representing a second local region around a second anchor point for a second image of the area.

5. The method of claim 4 , further comprising:
generating a first representation subspace for the first anchor point based on the first bag of descriptors; and
generating a second representation subspace for the second anchor point based on the second bag of descriptors.

6. The method of claim 5 , wherein detecting the absence or continued presence of the object at the property comprises:
performing feature matching across the first representation subspace and the second representation subspace;
computing a distance value that characterizes a detected feature match between corresponding descriptors of the first representation subspace and the second representation subspace; and
detecting the absence or continued presence of the object at the property based on the computed distance value.

7. The method of claim 2 , wherein detecting the absence or continued presence of the object at the property comprises:
detecting presence or absence of the object based on the set of anchor points and the set of descriptors for each anchor point in the set of anchor points.

8. The method of claim 1 , wherein performing the SIFT operation on the image comprises augmenting a plurality of attributes of the image, including augmenting attributes of the object and one or more items depicted in the image.

9. The method of claim 1 , wherein iteratively computing interest points for the object based on photometric augmentation comprises:
generating a plurality of representations of the image based on the augmenting of the plurality of attributes of the image; and
identifying a subset of features of the image that are repeated across the plurality of representations irrespective of the augmentation performed on the image.

10. A system comprising a processing device and non-transitory machine-readable storage medium storing instructions that are executable by the processing device to cause performance of operations comprising:
maintaining a plurality of images that were generated while monitoring an area of a property;
detecting, using at least some of the plurality of images, an object in a region of interest in the area;
for at least some images from the plurality of images:
iteratively computing interest points for the object based on photometric augmentation applied to the image before each iteration of computing the interest points, wherein the photometric augmentation comprises:
performing a Scale Invariant Feature Transform (SIFT) operation on the image;

generating a digital representation of the region and the object based on interest points that repeat across the plurality of images after each application of the photometric augmentation;
determining, based on the digital representation, a set of anchor points from the interest points that repeat across the plurality of images; and
using the set of anchor points, detecting an absence of the object in the area of the property or a continued presence of the object in the area of the property.

11. The system of claim 10 , wherein determining a set of anchor points comprises:
determining a respective set of descriptors for each anchor point in the set of anchor points, wherein each respective set of descriptors represents a local region around an interest point from which the corresponding anchor point was determined.

12. The system of claim 11 , wherein determining a set of anchor points from the interest points that repeat across the plurality of images comprises:
computing a respective probability value for each of the interest points that repeat across the plurality of images;
aggregating interest points having a respective probability value that exceeds a threshold probability value; and
determining the set of anchor points based on the aggregated interest points.

13. The system of claim 11 , wherein the operations further comprise:
determining a first bag of descriptors representing a first local region around a first anchor point for a first image of the area; and
determining a second bag of descriptors representing a second local region around a second anchor point for a second image of the area.

14. The system of claim 13 , wherein the operations further comprise:
generating a first representation subspace for the first anchor point based on the first bag of descriptors; and
generating a second representation subspace for the second anchor point based on the second bag of descriptors.

15. The system of claim 14 , wherein detecting the absence or continued presence of the object at the property comprises:
performing feature matching across the first representation subspace and the second representation subspace;
computing a distance value that characterizes a detected feature match between corresponding descriptors of the first representation subspace and the second representation subspace; and
detecting the absence or continued presence of the object at the property based on the computed distance value.

16. The system of claim 11 , wherein detecting the absence or continued presence of the object at the property comprises:
detecting presence or absence of the object based on the set of anchor points and the set of descriptors for each anchor point in the set of anchor points.

17. The system of claim 10 , wherein iteratively computing interest points for the object based on photometric augmentation co

Assignees

• Objectvideo Labs Llc

Inventors

• Taheri Sima
• Qian Gang
• Beach Allison

Classifications

• G06V20/20

  in augmented reality scenes · CPC title

• G06V10/46

  Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features (colour feature extraction G06V10/56) · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06V20/52 Primary

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title