Methods, systems, and media for evaluating images

Patent metadata

Field	Value
Publication number	US-11615643-B2
Application number	US-202117447067-A
Country	US
Kind code	B2
Filing date	Sep 8, 2021
Priority date	Dec 28, 2016
Publication date	Mar 28, 2023
Grant date	Mar 28, 2023

Abstract

A method may include obtaining an image including a face. The method may further include determining at least one time domain feature related to the face in the image and at least one frequency domain information related to the face in the image. The method may further include evaluating the quality of the image based on the at least one time domain feature and the frequency domain information.

First claim

What is claimed is:

1. A face recognition electronic system, comprising:
at least one storage device storing a set of instructions;
at least one processor in communication with the at least one storage device, wherein when executing the set of instructions the at least one processor:
obtain a two-dimensional (2D) image including a face, and three-dimensional (3D) data corresponding to the 2D image, the 2D image including a plurality of pixels, the 3D data including a plurality of points;
determine, for each of the plurality of pixels in the 2D image, a point in the 3D data corresponding to the pixel in the 2D image;
determine a mask image based on the point in the 3D data corresponding to each of the plurality of pixels in the 2D image;
determine, based on the mask image, a symmetry related parameter of the face in the 2D image; and
correct, based on the symmetry related parameter of the face, the 2D image to generate a corrected 2D image including a front view of the face.

2. The system of claim 1 , wherein to determine, for each of the plurality of pixels in the 2D image, the point in the 3D data corresponding to the pixel in the 2D image, the at least one processor:
determine at least one landmark point in the 2D image;
determine at least one point in the 3D data corresponding to the at least one landmark point in the 2D image;
determine a projection matrix based on the at least one point in the 3D data and a template 3D model of face;
determine a relationship between the 2D image and the 3D data based on the projection matrix;
determine, for each of the plurality of pixels in the 2D image, the point in the 3D data corresponding to the pixel in the 2D image based on the relationship between the 2D image and the 3D data.

3. The system of claim 2 , wherein the at least one landmark point in the 2D image includes at least one edge point of a facial feature.

4. The system of claim 2 , wherein to determine the at least one point in the 3D data corresponding to the at least one landmark point in the 2D image, the at least one processor:
identify coordinates of the at least one landmark point in the 2D image; and
identify points in the 3D data having same coordinates with the coordinates of the at least one landmark point in the 2D image as the at least one point in the 3D data corresponding to the at least one landmark point in the 2D image.

5. The system of claim 2 , wherein the template 3D model of face is generated based on a plurality of 2D images and corresponding 3D data.

6. The system of claim 1 , wherein to determine the symmetry related parameter of the face in the 2D image, the at least one processor:
divide the mask image into a first sub-image and a second sub-image;
determine a difference between the first sub-image and the second sub-image; and
determine, based on the difference between the first sub-image and the second sub-image, the symmetry related parameter of the face in the 2D image.

7. The system of claim 6 , wherein the at least one processor:
symmetrically divide the mask image along a vertical centerline into a left image and a right image, the left image being the first sub-image, the right image being the second sub-image.

8. The system of claim 6 , wherein to determine the difference between the first sub-image and the second sub-image, the at least one processor:
determine a first sum of values of pixels in the first sub-image and a second sum of values of pixels in the second sub-image; and
determine the difference between the first sub-image and the second sub-image based on the first sum and the second sum.

9. The system of claim 8 , wherein a value of a pixel in the mask image is associated with a normal line of a point in the 3D data corresponding to the pixel; and
in response to that a pixel in the mask image corresponding to a point in the 3D data of which the angle between the normal line and the Z direction is greater than 45°, the value of the pixel in the mask image is set as 1,
in response to that a pixel in the mask image corresponding to a point in the 3D data of which the angle between the normal line and the Z direction is equal or smaller than 45°, the value of the pixel is set as 0.

10. The system of claim 6 , wherein the difference between the first sub-image and the second sub-image is normalized.

11. The system of claim 1 , wherein to correct the 2D image, the at least one processor:
determine a Gaussian image based on the mask image;
determine a first coefficient associated with the symmetry related parameter, the mask image, and the Gaussian image;
flip the Gaussian image and the 2D image;
determine a second coefficient associated with the symmetry related parameter, the mask image, and the flipped Gaussian image;
determine a first matrix based on the 2D image and the Gaussian image;
determine a second matrix based on the 2D image and the first coefficient;
determine a third matrix based on the flipped 2D image and the second coefficient; and
correct the 2D image based on the first matrix, the second matrix, and the third matrix.

12. The system of claim 11 , wherein to correct the 2D image based on the first matrix, the second matrix, and the third matrix, the at least one processor:
determine a corrected matrix of the 2D image by adding up the first matrix, the second matrix, and the third matrix; and
correct the 2D image based on the corrected matrix of the 2D image.

13. A method implemented on at least one device each of which has at least one processor and storage, the method comprising:
obtaining a two-dimensional (2D) image including a face, and three-dimensional (3D) data corresponding to the 2D image, the 2D image including a plurality of pixels, the 3D data including a plurality of points;
determining, for each of the plurality of pixels in the 2D image, a point in the 3D data corresponding to the pixel in the 2D image;
determining a mask image based on the point in the 3D data corresponding to each of the plurality of pixels in the 2D image;
determining, based on the mask image, a symmetry related parameter of the face in the 2D image; and
correcting, based on the symmetry related parameter of the face, the 2D image to generate a corrected 2D image including a front view of the face.

14. The method of claim 13 , wherein the determining, for each of the plurality of pixels in the 2D image, the point in the 3D data corresponding to the pixel in the 2D image includes:
determining at least one landmark point in the 2D image;
determining at least one point in the 3D data corresponding to the at least one landmark point in the 2D image;
determining a projection matrix based on the at least one point in the 3D data and a template 3D model of face;
determining a relationship between the 2D image and the 3D data based on the projection matrix;
determining, for each of the plurality of pixels in the 2D image, the point in the 3D data corresponding to the at least one landmark point in the 2D image based on the relationship between the 2D image and the 3D data.

15. The method of claim 13 , wherein the determining of the symmetry related parameter of the face in the 2D image includes:
dividing the mask image into a first sub-image and a second sub-image;
determining a difference between the first sub-image and the second sub-image; and
determining, based on the difference between the first sub-image and the second sub-image, the symmetry related parameter of the face in the 2D image.

16. The method of claim 15 , wherein the dividing the mask image into a first sub-image and a sec

Assignees

• Zhejiang Dahua Technology Co

Inventors

• Guo Siyu
• Wang Haiyang
• Hao Jingsong
• Wang Gang

Classifications

• G06V10/431

  Frequency domain transformation; Autocorrelation · CPC title

• G06V40/161 Primary

  Detection; Localisation; Normalisation · CPC title

• G06V10/467

  Encoded features or binary features, e.g. local binary patterns [LBP] · CPC title

• G06V40/167 Primary

  using comparisons between temporally consecutive images · CPC title

• G06V10/993

  Evaluation of the quality of the acquired pattern · CPC title