Detection of manipulated images

Patent metadata

Field	Value
Publication number	US-11450152-B2
Application number	US-201816625951-A
Country	US
Kind code	B2
Filing date	Jun 29, 2018
Priority date	Jun 30, 2017
Publication date	Sep 20, 2022
Grant date	Sep 20, 2022

Abstract

An apparatus for detecting morphed or averaged images, wherein the morphed or averaged images are synthetically generated images including information from two or more different source images corresponding to two or more subjects. The apparatus may include a feature extraction module for receiving an input image and outputting a set of descriptor feature(s) characteristic of the image and a classifier module configured to allocate the input image either to a first class indicating that the image has been morphed or averaged or a second class indicating that it has not been morphed or averaged, based on the descriptor feature(s). The feature extraction module may include a plurality of neural networks providing complementary descriptor feature(s) to the classifier module. The apparatus further may include a fusion module for combining descriptor feature data from each neural network and transmitting the fused feature data to the classifier module.

First claim

The invention claimed is:

1. An apparatus for detecting morphed or averaged images, wherein each of the morphed or averaged images are synthetically generated images comprising information from two or more different source images corresponding to two or more subjects, the apparatus comprising:
a feature extraction module for receiving an input image and outputting a set of descriptor feature(s) characteristic of the image; and
a classifier module configured to allocate the input image either to a first class indicating that the image has been morphed or averaged or a second class indicating that it has not been morphed or averaged, based on the descriptor feature(s);
wherein the classifier module comprises a machine-learning system trained to classify single images using a training data set comprising morphed or averaged images and images that have not been morphed or averaged.

2. The apparatus as claimed in claim 1 , wherein the feature extraction module comprises a machine-learning system and the descriptor features depend on parameters determined from the use of a training data set comprising images.

3. The apparatus as claimed in claim 2 , wherein the training data set comprises morphed or averaged images and images that have not been morphed or averaged.

4. The apparatus as claimed in claim 1 , wherein the morphed or averaged images are morphed or averaged facial images.

5. The apparatus as claimed in claim 1 , further comprising an image pre-processing module arranged to extract and normalise a region of interest (for example a human face) from the image and to transmit the pre-processed image to the feature extraction module.

6. The apparatus as claimed in claim 1 , wherein the feature extraction module comprises a set of filters that are convolved with patches of the input image to provide a set of descriptor feature(s).

7. The apparatus as claimed in claim 6 , wherein the set of descriptor features comprises a string of binarised quantities derived from the convolutions, for example BSIF, as described herein.

8. The apparatus as claimed in claim 1 , wherein the classifier module comprises a linear support vector machine or a probabilistic collaborative representation classifier.

9. The apparatus as claimed in claim 1 , wherein the feature extraction module comprises at least one convolutional neural network.

10. The apparatus as claimed in claim 9 , wherein the feature extraction module comprises a plurality of substantially statistically independent neural networks providing complementary descriptor feature(s) to the classifier module.

11. The apparatus as claimed in claim 9 , wherein the neural network(s) comprise deep convolutional neural network(s), preferably having three or more convolutional layers.

12. The apparatus as claimed in claim 11 , wherein the descriptor features are extracted from the first fully-connected layer of each deep convolutional neural network.

13. The apparatus as claimed in claim 10 , wherein the apparatus further comprises a feature level fusion module for combining descriptor feature data from each neural network and transmitting the fused (e.g. concatenated) feature data to the classifier module.

14. The apparatus as claimed in claim 9 , wherein the neural network(s) are individually trained using a set of images comprising morphed or averaged images and images that have not been morphed or averaged in order to train their filters to provide descriptor features suited for determining whether an image has been morphed or averaged.

15. An apparatus for detecting morphed or averaged images, wherein each of the morphed or averaged images are synthetically generated images comprising information from two or more different source images corresponding to two or more subjects, the apparatus comprising:
a feature extraction module for receiving an input image and outputting a set of descriptor feature(s) characteristic of the image; and
a classifier module configured to allocate the input image either to a first class indicating that the image has been morphed or averaged or a second class indicating that it has not been morphed or averaged, based on the descriptor feature(s); wherein
the feature extraction module comprises a plurality of neural networks providing complementary descriptor feature(s) to the classifier module;
the apparatus further comprises a fusion module for combining descriptor feature data from each neural network and transmitting the fused feature data to the classifier module; and
the classifier module comprises a machine-learning system trained to classify the images using a training data set comprising morphed or averaged images and images that have not been morphed or averaged.

16. The apparatus as claimed in claim 15 , wherein the neural networks are different deep convolutional neural networks.

17. The apparatus as claimed in claim 15 , wherein the neural networks are substantially statistically independent.

18. The apparatus as claimed in claim 15 , wherein the morphed or averaged images are morphed or averaged facial images.

19. The apparatus as claimed in claim 15 , further comprising the features of claim 2 .

20. A method of detecting morphed or averaged images, wherein each of the morphed or averaged images are synthetically generated images comprising information from two or more different source images corresponding to two or more subjects, the apparatus comprising:
receiving an input image;
generating a set of descriptor feature(s) characteristic of that image;
classifying the image by allocating it to either to a first class indicating that the image has been morphed or averaged or a second class indicating that it has not been morphed or averaged, based on the descriptor feature(s);
wherein the classification step comprises using a machine-learning system trained to classify single images using a training data set comprising morphed or averaged images and images that have not been morphed or averaged.

21. The method as claimed in claim 20 , wherein the step of generating the descriptor feature(s) comprises using a plurality of preferably substantially statistically independent neural networks providing complementary descriptor feature(s) to the classifier module; and descriptor feature data from each neural network is combined prior to classification.

22. The method according to claim 20 , further comprising the step of training a classifier used in the classifying step.

23. The method according to claim 20 , comprising training a feature extraction module that generates the descriptor feature(s) using a set of images comprising morphed or averaged images and images that have not been morphed or averaged in order to train the module to provide descriptor features suited for determining whether an image has been morphed or averaged.

24. A computer programme product comprising instructions that when executed on a computer configure the computer to perform the method of claim 20 .

Assignees

• Norwegian Univ Of Science And Technology
• Norwegian Univ Sci & Tech Ntnu

Inventors

• Ramachandra Raghavendra
• Raja Kiran Bylappa
• Venkatesh Sushma
• Busch Christoph

Classifications

• G06V40/16

  Human faces, e.g. facial parts, sketches or expressions · CPC title

• G06V10/806

  of extracted features · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06F21/32 Primary

  using biometric data, e.g. fingerprints, iris scans or voiceprints · CPC title