Detecting hybdrid-distance adversarial patches

Patent metadata

Field	Value
Publication number	US-11875489-B2
Application number	US-202117363054-A
Country	US
Kind code	B2
Filing date	Jun 30, 2021
Priority date	Jun 30, 2021
Publication date	Jan 16, 2024
Grant date	Jan 16, 2024

Abstract

A hybrid-distance adversarial patch generator can be trained to generate a hybrid adversarial patch effective at multiple distances. The hybrid patch can be inserted into multiple sample images, each depicting an object, to simulate inclusion of the hybrid patch at multiple distances. The multiple sample images can then be used to train an object detection model to detect the objects.

First claim

What is claimed is:

1. A method, comprising:
training a hybrid adversarial patch generator;
generating, via the hybrid adversarial patch generator, a hybrid adversarial patch;
inserting the hybrid adversarial patch into a first image, resulting in a first modified image depicting the hybrid adversarial patch at a first distance and a first object; and
inserting the hybrid adversarial patch into a second image, resulting in a second modified image depicting the hybrid adversarial patch at a second distance and a second object;
submitting the first image and the second image to a first object detection model;
receiving an output from the first object detection model; and
updating the hybrid adversarial patch generator based on the output.

2. The method of claim 1 , further comprising:
training a near adversarial patch generator, the training the near adversarial patch generator including updating a first weight by a first amount; and
training a far adversarial patch generator, the training the far adversarial patch generator including updating the first weight by a second amount, wherein the training the hybrid adversarial patch generator includes updating the first weight by a third amount, the third amount based on the first amount and the second amount.

3. The method of claim 1 , wherein the training the near adversarial patch generator includes training an adversarial patch modifier to modify patches generated by a pretrained patch generator.

4. The method of claim 1 , further comprising training the first object detection model to detect:
the first object in the first modified image; and
the second object in the second modified image, wherein the training the first object detection model includes:
submitting the first modified image to the first object detection model;
receiving a first output from the first object detection model as a result of the submitting the first modified image;
submitting the second modified image to the first object detection model;
receiving a second output from the first object detection model as a result of the submitting the second modified image;
evaluating a performance of the first object detection model based on the first output and the second output; and
updating the first object detection model based on the performance.

5. The method of claim 1 , wherein the generating the hybrid adversarial patch includes:
receiving a near adversarial patch;
receiving a far adversarial patch; and
merging the near adversarial patch and the far adversarial patch into the hybrid adversarial patch.

6. The method of claim 5 , wherein the merging includes:
determining a first Red-Green-Blue (RGB) value of a first pixel of the near adversarial patch, the first pixel in a first relative location in the near adversarial patch;
determining a second RGB value of a second pixel of the far adversarial patch, the second pixel in the first relative location in the far adversarial patch; and
generating a third RGB value of a third pixel of the hybrid adversarial patch, the generating based on the first RGB value and the second RGB value, the third pixel in the first location in the hybrid adversarial patch.

7. The method of claim 1 , wherein the first object is the second object.

8. A system, comprising:
a memory; and
a processor coupled to the memory, the processor configured to:
train a hybrid adversarial patch generator;
generate, via the hybrid adversarial patch generator, a hybrid adversarial patch;
insert the hybrid adversarial patch into a first image, resulting in a first modified image depicting the hybrid adversarial patch at a first distance and a first object;
insert the hybrid adversarial patch into a second image, resulting in a second modified image depicting the hybrid adversarial patch at a second distance and a second object;
submit the first image and the second image to a first object detection model;
receive an output from the first object detection model; and
update the hybrid adversarial patch generator based on the output.

9. The system of claim 8 , wherein the processor is further configured to:
train a near adversarial patch generator, the training the near adversarial patch generator including updating a first weight by a first amount; and
train a far adversarial patch generator, the training the far adversarial patch generator including updating the first weight by a second amount, wherein the training the hybrid adversarial patch generator includes updating the first weight by a third amount, the third amount based on the first amount and the second amount.

10. The system of claim 8 , wherein the training the near adversarial patch generator includes training an adversarial patch modifier to modify patches generated by a pretrained patch generator.

11. The system of claim 8 , wherein the processor is further configured to train the first object detection model to detect:
the first object in the first modified image; and
the second object in the second modified image, wherein the training the first object detection model includes:
wherein the training the first object detection model includes:
submitting the first modified image to the first object detection model;
receiving a first output from the first object detection model as a result of the submitting the first modified image;
submitting the second modified image to the first object detection model;
receiving a second output from the first object detection model as a result of the submitting the second modified image;
evaluating a performance of the first object detection model based on the first output and the second output; and
updating the first object detection model based on the performance.

12. The system of claim 8 , wherein the generating the hybrid adversarial patch includes:
receiving a near adversarial patch;
receiving a far adversarial patch; and
merging the near adversarial patch and the far adversarial patch into the hybrid adversarial patch.

13. The system of claim 12 , wherein the merging includes:
determining a first Red-Green-Blue (RGB) value of a first pixel of the near adversarial patch, the first pixel in a first relative location in the near adversarial patch;
determining a second RGB value of a second pixel of the far adversarial patch, the second pixel in the first relative location in the far adversarial patch; and
generating a third RGB value of a third pixel of the hybrid adversarial patch, the generating based on the first RGB value and the second RGB value, the third pixel in the first location in the hybrid adversarial patch.

14. The system of claim 8 , wherein the first object is the second object.

15. A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
train a hybrid adversarial patch generator;
generate, via the hybrid adversarial patch generator, a hybrid adversarial patch;
insert the hybrid adversarial patch into a first image, resulting in a first modified image depicting the hybrid adversarial patch at a first distance and a first object;
insert the hybrid adversarial patch into a second image, resulting in a second modified image depicting the hybrid adversarial patch at a second distance and a second object;
submit the first image and the second image to a first object detection model;
receive an output from the first object detection model; and
update the hybrid adversarial patch generator based on the output.

Assignees

• IBM

Inventors

• Fan Quanfu
• Liu Sijia
• Chen Richard
• Panda Rameswar

Classifications

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T2207/20081

  Training; Learning · CPC title