Adversarial masks for scene-customized false detection removal

Patent metadata

Field	Value
Publication number	US-12387480-B2
Application number	US-202418619370-A
Country	US
Kind code	B2
Filing date	Mar 28, 2024
Priority date	Nov 12, 2020
Publication date	Aug 12, 2025
Grant date	Aug 12, 2025

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving multiple images from a camera, each image of the multiple images representative of a detection of an object within the image. For each image of the multiple images the methods include: determining a set of detected objects within the image, each object defined by a respective bounding box, and determining, from the set of detected objects within the image and ground truth labels, a false detection of a first object. The methods further include determining that a target object threshold is met based on a number of false detections of the first object in the multiple images, generating, based on the number of false detections for the first object meeting the target object threshold, an adversarial mask for the first object, and providing, to the camera, the adversarial mask.

First claim

What is claimed is:

1. A computer-implemented method comprising:
accessing a set of target regions for a camera;
accessing a set of images that were captured by the camera;
initializing, for a first image of the set of images, an adversarial mask including a first value for an adversarial mask parameter for the set of target regions;
generating, for a second image of the set of images, an updated adversarial mask, the generating including:
applying the adversarial mask to the second image, and
updating the first value to a second value for the adversarial mask parameter using a result of applying the adversarial mask to the second image; and

providing, to the camera, the updated adversarial mask for use by an object detection model.

2. The method of claim 1 , comprising accessing data representing the set of target regions that indicate regions within a field of view of the camera in which there was a false positive detection.

3. The method of claim 1 , comprising detecting, from a plurality of regions within a field of view of the camera, the set of target regions that are associated with false positive detections.

4. The method of claim 1 , comprising:
detecting, in a first image, a first bounding box for a first object that was a false positive detection;
detecting, in a second image, a second bounding box for a second object that was a true positive detection;
determining that the first bounding box and the second bounding box overlap in an image region represented by the same coordinates in the respective images; and
assigning the image region as a target region in the set of target regions that are associated with false positive detections.

5. The method of claim 1 , wherein initializing the adversarial mask comprises determining an altered version of the first image.

6. The method of claim 1 , wherein initializing the adversarial mask including the first value for the adversarial mask parameter comprises:
performing a forward pass on the adversarial mask using a fast gradient sign method;
performing a backward pass on the adversarial mask using the fast gradient sign method;
determining a sign of a gradient of a loss function associated with an object detection model; and
using the sign of the gradient as an initialization value of a noise mask learnable parameter for the adversarial mask.

7. The method of claim 1 , wherein applying the adversarial mask to the second image comprises:
performing a forward pass on the second image using a fast gradient sign method;
performing a backward pass on the second image using the fast gradient sign method;
determining, using the forward and backward passes, an object detection loss; and
determining, using the object detection loss, a noise mask learnable parameter of the updated adversarial mask.

8. A system comprising one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
accessing by the one or more computers, a set of target regions for a camera;
accessing, by the one or more computers, a set of images from the camera;
initializing, for a first image of the set of images, an adversarial mask including a first value for an adversarial mask parameter for the set of target regions;
generating, for a second image of the set of images, an updated adversarial mask, the generating including:
applying the adversarial mask to the second image, and
updating the first value to a second value for the adversarial mask parameter using a result of applying the adversarial mask to the second image; and

providing, to the camera, the updated adversarial mask for use by an object detection model.

9. The system of claim 8 , wherein the operations comprise accessing data representing the set of target regions that indicate regions within a field of view of the camera that are associated with false positive detections.

10. The system of claim 8 , wherein the operations comprise detecting, from a plurality of regions within a field of view of the camera, the set of target regions that are associated with false positive detections.

11. The system of claim 8 , wherein:
the set of images comprise at least a first image that includes a first representation of a first object and a second representation of a second object,
a first bounding box for the first representation of the first object overlaps a second bounding box for the second representation of the second object,
and the second object is an object of interest.

12. The system of claim 8 , wherein initializing the adversarial mask comprises determining an altered version of the first image.

13. The system of claim 8 , wherein initializing the adversarial mask including a first adversarial mask parameter comprises:
performing a forward pass on the adversarial mask using a fast gradient sign method;
performing a backward pass on the adversarial mask using the fast gradient sign method;
determining a sign of a gradient of a loss function associated with an object detection model; and
using the sign of the gradient as an initialization value of a noise mask learnable parameter for the adversarial mask.

14. The system of claim 8 , wherein applying the adversarial mask to the second image comprises:
performing a forward pass on the second image using a fast gradient sign method;
performing a backward pass on the second image using the fast gradient sign method;
determining, using the forward and backward passes, an object detection loss; and
determining, using the object detection loss, a noise mask learnable parameter of the updated adversarial mask.

15. The system of claim 8 , wherein the operations comprise:
detecting, in a first image, a first bounding box for a first object that was a false positive detection;
detecting, in a second image, a second bounding box for a second object that was a true positive detection;
determining that the first bounding box and the second bounding box overlap in an image region represented by the same coordinates in the respective images; and
assigning the image region as a target region in the set of target regions that are associated with false positive detections.

16. One or more non-transitory computer storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform the following operations:
accessing by the one or more computers, a set of target regions for a camera;
accessing, by the one or more computers, a set of images from the camera;
initializing, for a first image of the set of images, an adversarial mask including a first value for an adversarial mask parameter for the set of target regions;
generating, for a second image of the set of images, an updated adversarial mask, the generating including:
applying the adversarial mask to the second image, and
updating the first value to a second value for the adversarial mask parameter using a result of applying the adversarial mask to the second image; and

providing, to the camera, the updated adversarial mask for use by an object detection model.

17. The non-transitory computer storage media of claim 16 , wherein the operations comprise accessing data representing the set of target regions that indicate regions within a field of view of the camera that are associated with false positive detections.

18. The non-transitory computer storage media of claim 16 , wherein the operations comprise d

Assignees

• Objectvideo Labs Llc

Inventors

• Beach Allison
• Qian Gang
• Carmena Eduardo Romera

Classifications

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/70

  using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06V20/20

  in augmented reality scenes · CPC title