Method and system of multi-attribute network based fake imagery detection (MANFID)

Patent metadata

Field	Value
Publication number	US-12347086-B2
Application number	US-202217876908-A
Country	US
Kind code	B2
Filing date	Jul 29, 2022
Priority date	Jul 29, 2021
Publication date	Jul 1, 2025
Grant date	Jul 1, 2025

Abstract

A method for detecting fake images includes: obtaining an image for authentication, and hand-crafting a multi-attribute classifier to determine whether the image is authentic. Hand-crafting the multi-attribute classifier includes fusing at least an image classifier, an image spectrum classifier, a co-occurrence matrix classifier, and a one-dimensional (1D) power spectrum density (PSD) classifier. The multi-attribute classifier is trained by pre-processing training images to generate an attribute-specific training dataset to train each of the image classifier, the image spectrum classifier, the co-occurrence matrix classifier, and the 1D PSD classifier.

First claim

What is claimed is:

1. A method for detecting fake images, comprising:
obtaining an image for authentication; and
hand-crafting a multi-attribute classifier to determine whether the image is authentic;
wherein:
hand-crafting the multi-attribute classifier includes fusing at least an image classifier, an image spectrum classifier, a co-occurrence matrix classifier, and a one-dimensional (1D) power spectrum density (PSD) classifier by performing:
simultaneously performing neuron pruning and feature selection in a fusion neural network while optimizing weights of the fusion neural network using a sparse group lasso algorithm during a training stage of the fusion neural network; and

the multi-attribute classifier is trained by pre-processing training images to generate an attribute-specific training dataset to train each of the image classifier, the image spectrum classifier, the co-occurrence matrix classifier, and the 1D PSD classifier.

2. The method according to claim 1 , wherein:
each of the image classifier, the image spectrum classifier, and the co-occurrence matrix classifier includes a multi-layer convolutional neural network (CNN); and
the 1D PSD classifier includes a single layer CNN.

3. The method according to claim 2 , wherein:
the single layer CNN includes a 1×1×80 convolutional layer; and
the multi-layer CNN includes a 128×128×3 convolutional layer, a 128×128×16 convolutional layer, a 64×64×32 convolutional layer, a 32×32×64 convolutional layer, a 16×16×128 convolutional layer, a 8×8×256 convolutional layer, a 4×4×512 convolutional layer, and a 1×1×512 convolutional layer, that are cascaded together.

4. The method according to claim 1 , wherein pre-processing the training images to generate the attribute-specific training dataset to train each of the image classifier, the image spectrum classifier, the co-occurrence matrix classifier, and the 1D PSD classifier includes:
performing an augmentation process on the training images to obtain a set of augmented training images; and
performing a mixup process on the set of augmented training images to generate the attribute-specific training dataset to train the image classifier.

5. The method according to claim 1 , wherein pre-processing the training images to generate the attribute-specific training dataset to train each of the image classifier, the image spectrum classifier, the co-occurrence matrix classifier, and the 1D PSD classifier includes:
performing an augmentation process on the training images to obtain a set of augmented training images;
performing a mixup process on the set of augmented training images to obtain a set of mixup training images; and
performing a discrete Fourier transform (DFT) process on the set of mixup training images to generate the attribute-specific training dataset to train the image spectrum classifier.

6. The method according to claim 1 , wherein pre-processing the training images to generate the attribute-specific training dataset to train each of the image classifier, the image spectrum classifier, the co-occurrence matrix classifier, and the 1D PSD classifier includes:
performing an augmentation process on the training images to obtain a set of augmented training images;
performing a mixup process on the set of augmented training images to obtain a set of mixup training images; and
computing co-occurrence matrices directly on pixels on the set of mixup training images on each of the red, green and blue channels to generate the attribute-specific training dataset to train the co-occurrence matrix classifier.

7. The method according to claim 1 , wherein pre-processing the training images to generate the attribute-specific training dataset to train each of the image classifier, the image spectrum classifier, the co-occurrence matrix classifier, and the 1D PSD classifier includes:
performing an augmentation process on the training images to obtain a set of augmented training images;
performing a mixup process on the set of augmented training images to obtain a set of mixup training images;
performing a discrete Fourier transform on the set of mixup training images to obtain a set of two-dimensional (2D) amplitude spectrums of the set of mixup training images; and
performing an azimuthal averaging of the set of 2D amplitude spectrums to generate the attribute-specific training dataset to train the 1D PSD classifier.

8. The method according to claim 4 , wherein the augmentation process includes:
rotation, translation, cropping, resizing, JPEG compression, flipping, blurring, random erasing, or a combination thereof.

9. The method according to claim 4 , wherein the mixup process includes:
random convex combination of raw inputs; and
convex combination of one-hot label encodings.

10. A system for detecting fake images, comprising:
a memory storing computer program instructions; and
a processor coupled to the memory and, when executing the computer program instructions, configured to perform:
obtaining an image for authentication; and
hand-crafting a multi-attribute classifier to determine whether the image is authentic;

wherein:
hand-crafting the multi-attribute classifier includes fusing at least an image classifier, an image spectrum classifier, a co-occurrence matrix classifier, and a one-dimensional (1D) power spectrum density (PSD) classifier by performing:
simultaneously performing neuron pruning and feature selection in a fusion neural network while optimizing weights of the fusion neural network using a sparse group lasso algorithm during a training stage of the fusion neural network; and

the multi-attribute classifier is trained by pre-processing training images to generate an attribute-specific training dataset to train each of the image classifier, the image spectrum classifier, the co-occurrence matrix classifier, and the 1D PSD classifier.

11. The system according to claim 10 , wherein:
each of the image classifier, the image spectrum classifier, and the co-occurrence matrix classifier includes a multi-layer convolutional neural network (CNN); and
the 1D PSD classifier includes a single layer CNN.

12. The system according to claim 11 , wherein:
the single layer CNN includes a 1×1×80 convolutional layer; and
the multi-layer CNN includes a 128×128×3 convolutional layer, a 128×128×16 convolutional layer, a 64×64×32 convolutional layer, a 32×32×64 convolutional layer, a 16×16×128 convolutional layer, a 8×8×256 convolutional layer, a 4×4×512 convolutional layer, and a 1×1×512 convolutional layer, that are cascaded together.

13. The system according to claim 10 , wherein pre-processing the training images to generate the attribute-specific training dataset to train each of the image classifier, the image spectrum classifier, the co-occurrence matrix classifier, and the 1D PSD classifier includes:
performing an augmentation process on the training images to obtain a set of augmented training images; and
performing a mixup process on the set of augmented training images to generate the attribute-specific training dataset to train the image classifier.

14. The system according to claim 10 , wherein pre-processing the training images to generate the attribute-specific training dataset to train each of the image classifier, the image spectrum classifier, the co-occurrence matrix classifier, and the 1D PSD classifier includes:
performing an augmentation process on the training images to obtain a set of augmented training images;
performing a mixup process on the set of augmented training images to obtain a set of mixup training images; and
performing a discrete Fourier transform (D

Assignees

• Intelligent Fusion Tech Inc

Inventors

• Chen Hua-Mei
• Sul Bora
• Chen Genshe
• Blasch Erik
• Pham Khanh

Classifications

• G06V10/82 Primary

  using neural networks · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06V10/811

  the classifiers operating on different input data, e.g. multi-modal recognition · CPC title

• G06T7/00

  Image analysis · CPC title