Training an object classifier with a known object in images of unknown objects

Patent metadata

Field	Value
Publication number	US-12340559-B2
Application number	US-202217903484-A
Country	US
Kind code	B2
Filing date	Sep 6, 2022
Priority date	Sep 16, 2021
Publication date	Jun 24, 2025
Grant date	Jun 24, 2025

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for objection classification. One of the methods includes: obtaining a first set of images of objects that have a likelihood of being at a property that satisfies a likelihood threshold; generating, for each object, a binary classifier from a set of images of the respective object; determining, using at least one of the binary classifiers, that an image of an unknown object was classified as an object from the objects; in response to determining, using the binary classifiers, that the image of the unknown object was classified as an object from the objects, selecting a second set of images of unknown objects that does not include the image; and generating a multiclass classifier for use classifying objects using i) the first set as respective classes and ii) the second set that does not include the image.

First claim

What is claimed is:

1. A computer implemented method, comprising:
obtaining a first set of images of one or more objects, each of which has a likelihood of being at a property that satisfies a likelihood threshold;
generating, for each object from the one or more objects each of which has a likelihood of being at a property that satisfies a likelihood threshold, a binary classifier from a set of images of the respective object that outputs a binary value indicating whether an image depicting an object satisfies a corresponding criterion that is specific to the binary classifier, the generating comprising generating two or more binary classifiers that a) are each for an object classification and b) indicate whether a corresponding image satisfies a different corresponding criterion for the corresponding object classification;
determining, using at least one of the binary classifiers, that an image of an unknown object of a plurality of unknown objects was classified as an object from the one or more objects each of which has the respective likelihood of being at the property that satisfies the likelihood threshold;
in response to determining, using at least one of the binary classifiers, that the image of the unknown object of the plurality of unknown objects was classified as an object from the one or more objects each of which has the respective likelihood of being at the property that satisfies the likelihood threshold, selecting a second set of images of the unknown objects from the plurality of unknown objects that does not include the image; and
generating a multiclass classifier that outputs a non-binary value indicating whether an image depicting another object satisfies one of a plurality of criterion to classify objects depicted in images of the property using i) the first set of images of the one or more objects as respective classes and ii) the second set of images of the unknown objects that does not include the image.

2. The method of claim 1 , wherein selecting the second set of images comprises selecting the second set of images of the unknown objects from the plurality of unknown objects that does not include any images of the object.

3. The method of claim 1 , wherein the one or more objects each of which has a likelihood of being at a property comprise one or more residents.

4. The method of claim 1 , further comprising obtaining images of the plurality of unknown objects, wherein selecting the second set of images comprises selecting the second set of images from the images.

5. The method of claim 1 , wherein generating a multiclass classifier further comprises using the first set of images as positive examples and the second set of images from the images of unknown objects as negative examples.

6. The method of claim 1 , wherein the one or more objects comprises at least one of one or more people, one or more animals, or one or more vehicles.

7. The method of claim 1 , further comprising providing the multiclass classifier to a device for use in classifying objects as known objects.

8. The method of claim 1 , further comprising:
receiving an image of an object;
classifying, using a multiclass classifier that was trained to classify objects as known or unknown objects with i) a first set of images of one or more known objects each of which has a) a likelihood of being at the property that satisfies the threshold likelihood and b) a binary classifier that indicates whether an image likely depicts the respective known object and ii) a second set of images of the unknown objects that does not include an image that was classified by one of the one or more binary classifiers as likely depicting a respective known object, the object as a known object or unknown object; and
in response to classifying the object as a known object or unknown object, performing an automated action for known objects or unknown objects, respectively.

9. The method of claim 1 , further comprising:
validating at least one of the binary classifiers by determining at least one false positive within the positive sample from a confusion matrix for the at least one binary classifier;
updating the first set of images by removing the at least one false positive from the first set of images; and
generating a new multiclass classifier using i) the updated, first set of images of the one or more objects as respective classes and ii) the second set of images of the unknown objects that does not include the image.

10. The method of claim 1 , wherein the plurality of unknown objects comprise objects for which a binary classifier is not associated at the property.

11. A system comprising one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
obtaining a first set of images of one or more objects, each of which has a likelihood of being at a property that satisfies a likelihood threshold;
generating, for each object from the one or more objects each of which has a likelihood of being at a property that satisfies a likelihood threshold, a binary classifier from a set of images of the respective object that outputs a binary value indicating whether an image depicting an object satisfies a corresponding criterion that is specific to the binary classifier, the generating comprising generating two or more binary classifiers that a) are each for an object classification and b) indicate whether a corresponding image satisfies a different corresponding criterion for the corresponding object classification;
determining, using at least one of the binary classifiers, that an image of an unknown object of a plurality of unknown objects was classified as an object from the one or more objects each of which has the respective likelihood of being at the property that satisfies the likelihood threshold;
in response to determining, using at least one of the binary classifiers, that the image of the unknown object of the plurality of unknown objects was classified as an object from the one or more objects each of which has the respective likelihood of being at the property that satisfies the likelihood threshold, selecting a second set of images of unknown objects from the plurality of unknown objects that does not include the image; and
generating a multiclass classifier that outputs a non-binary value indicating whether an image depicting another object satisfies one of a plurality of criterion to classify objects depicted in images of the property using i) the first set of images of the one or more objects as respective classes and ii) the second set of images of the unknown objects that does not include the image.

12. The system of claim 11 , wherein selecting the second set of images comprises selecting the second set of images of the unknown objects from the plurality of unknown objects that does not include any images of the object.

13. The system of claim 11 , wherein the one or more objects each of which has a likelihood of being at a property comprise one or more residents.

14. The system of claim 11 , the operations further comprising obtaining images of the plurality of unknown objects, wherein selecting the second set of images comprises selecting the second set of images from the images.

15. The system of claim 11 , wherein generating a multiclass classifier further comprises using the first set of images as positive examples and the second set of images from the images of unknown objects as negative examples.

16. The system of claim 11 , wherein the one or mor

Assignees

• Objectvideo Labs Llc

Inventors

• Wu Yuanwei
• Qian Gang
• Beach Allison

Classifications

• G06V10/771

  Feature selection, e.g. selecting representative features from a multi-dimensional feature space · CPC title

• G06V10/776

  Validation; Performance evaluation · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title

• G06V10/764 Primary

  using classification, e.g. of video objects · CPC title