Image analysis system for forensic facial comparison

Patent metadata

Field	Value
Publication number	US-12475737-B2
Application number	US-202117909268-A
Country	US
Kind code	B2
Filing date	Mar 5, 2021
Priority date	Mar 6, 2020
Publication date	Nov 18, 2025
Grant date	Nov 18, 2025

Abstract

An automatic forensic facial comparison system, FFC, having a questioned image (I 1 ) and a reference image (I 2 ), captured by means of acquisition of images of a subject, comprising processing means configured to carry out FFC steps: at least one morphological analysis stage ( 11 ), mandatory, and optionally a holistic comparison stage ( 12 ), and/or an image overlay stage ( 13 ), and/or a photo-anthropometry stage ( 14 ), and/or a decision-making stage ( 15 ). For each stage ( 11, 12, 13, 14 ) corresponding to FFC methods, the processing means calculate an overall indicator value of the stage carried out. In the last decision-making stage ( 15 ), the processing means calculate a fuzzy value by applying soft computing, obtained as a sum of the overall indicator value of each stage previously carried out ( 11, 12, 13, 14 ), each value being weighted by a weight based on a set of data to support the decision-making stage ( 15 ) indicative of a degree of reliability of each stage and of the quality of the starting images (I 1 , I 2 ).

First claim

The invention claimed is:

1 . An image analysis system for forensic facial comparison, comprising:
image acquisition means for acquiring at least one questioned image (I 1 ) of a subject, processing means with access to the questioned image (I 1 ) and at least one reference image (I 2 );
wherein the processing means automatically perform the forensic facial comparison by comparing the questioned image (I 1 ) with the reference image (I 2 ), wherein the processing means are configured to perform at least one stage of morphological analysis (I 1 ) of forensic facial comparison by means of neural networks, and a decision-making stage (I 5 ) to obtain a fuzzy value applying soft computing, wherein the fuzzy value is an indicator of the probability that the questioned image (I 1 ) and reference image (I 2 ) correspond to the same subject; and
wherein the morphological analysis step (I 1 ) comprises:
labelling by means of a first neural network the questioned image (I 1 ) and the reference image (I 2 ) with morphological descriptors, said morphological descriptors representing anatomical and/or morphological aspects of a face; comparing the morphological descriptors labelled within the questioned facial image (I 1 ) and within the reference facial image (I 2 ) and calculating a degree of similarity between the morphological descriptors labelled within the questioned facial image (I 1 ) and within the reference facial image (I 2 ), obtaining a value x11;
detecting by means of a second neural network individualizing elements in the questioned image (I 1 ) and in the reference image (I 2 ), comparing the individualizing elements detected in the questioned facial image (I 1 ) and in the reference facial image (I 2 ) and obtaining a degree of similarity between the individualizing elements detected in the questioned facial image (I 1 ) and in the reference facial image (I 2 ), obtaining a value x12;
calculating a degree of asymmetry between the questioned image (I 1 ) and the reference image (I 2 ), obtaining a value x13, by calculating the difference between a degree of symmetry of the questioned image (I 1 ) and a degree of symmetry of the reference image (I 2 ), the degree of symmetry of the questioned image (I 1 ) calculated as the difference between the questioned image (I 1 ) and its mirror image, and the degree of symmetry of the reference image (I 2 ) calculated as the difference between the reference image (I 2 ) and its mirror image;
obtaining a value, E1, which is an overall indicator of the forensic facial comparison stage of morphological analysis (I 1 ), wherein E1=x11*w11+x12*w12+x13 w13, wherein w11, w12 and w13 are configurable weights that meet w12>w11>w13.

2 . The system according to claim 1 , wherein the first neural network is a multiclass network and the second neural network is a multi-task network.

3 . The system according to claim 1 , wherein the weights of the morphological analysis stage (I 1 ) are w12=0.6, w11=0.3 and w13=0.1.

4 . The system according to claim 1 , wherein the processing means are configured to also perform a holistic comparison step (I 2 ) of forensic facial comparison using siamese networks, which provide a value, E2, overall indicator of the holistic comparison stage (I 2 ) that indicates a degree of confidence with which the questioned and reference images (I 1 , I 2 ) show the same subject.

5 . The system according to claim 1 , wherein the processing means are configured to further perform a step of superimposing facial images (I 3 ) for forensic facial comparison comprising:
applying a transformation, of similarity if the two questioned and reference images (I 1 , I 2 ) are two-dimensional, or projective if at least one of the two questioned and reference images (I 1 , I 2 ) is three-dimensional, to align the two questioned images and reference (I 1 , I 2 ); segmenting silhouettes and facial regions in the questioned and reference images (I 1 , I 2 ) using a deformable segmentation model or a deep neural network; comparing the silhouettes and segmented facial regions, to obtain a value Yi indicating the correspondence between the questioned image (I 1 ) and the reference image (I 2 ) in each facial region i; obtaining a value, E3, which is an overall indicator of the facial image overlay stage (I 3 ), wherein E3=ΣYi*wi, wherein i denotes a facial region and wi a configurable weight assigned to the facial region i.

6 . The system according to claim 1 , wherein the processing means are configured to also perform a photo-anthropometry step ( 14 ) of forensic facial comparison comprising:
detecting marks in the two questioned and reference images (I 1 , I 2 ) using deep neural networks and estimating two-dimensional indices of the questioned image (I 1 ) and the reference image (I 2 ) by calculating a Euclidean distance between the marks detected in the questioned image (I 1 ) and in the reference image (I 2 ) respectively; estimating three-dimensional indices of the questioned image (I 1 ) and the reference image (I 2 ) by calculating a least squares linear regression starting from the estimated two-dimensional indices of the questioned image (I 1 ) and the reference image (I 2 ) respectively; obtaining a value, E4, which is an overall indicator of the photo-anthropometry stage ( 14 ), by comparing the indices in three dimensions estimated from the questioned image (I 1 ) with the indices in three dimensions estimated from the reference image (I 2 ).

7 . The system according to claim 1 , wherein the processing means are configured to, in the decision-making stage ( 15 ), obtain the fuzzy value as the sum of the overall indicator value of each forensic facial comparison stage performed by the processing means, each value used in the sum weighted by a defined weight based on a set of data to support the decision-making stage ( 15 ) indicative of a degree of reliability of the forensic facial comparison stage performed, and indicative of the quality of the questioned and reference images (I 1 , I 2 ).

8 . The system according to claim 1 , wherein the questioned image (I 1 ) and the reference image (I 2 ) have the same dimensionality, in two dimensions or in three dimensions.

9 . The system according to claim 1 , wherein the questioned image (I 1 ) and the reference image (I 2 ) have a different dimensionality, in two dimensions or three dimensions.

Assignees

• Univ Granada
• Panacea Cooperative Res S Coop

Inventors

• Martos Fernández Rubén
• Bermejo Nievas Enrique
• Ibáñez Panizo Oscar
• Valsecchi Andrea
• Mesejo Santiago Pablo
• Vasconcellos Da Silva Alexsandro
• Navarro Merino Fernando
• Alemán Aguilera María Inmaculada
• Cordón García Oscar
• Damas Arroyo Sergio

Classifications

• G06V10/34

  Smoothing or thinning of the pattern; Morphological operations; Skeletonisation · CPC title

• G06V10/26

  Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion · CPC title

• G06V10/16

  using multiple overlapping images; Image stitching · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V20/70

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title