Method and apparatus with liveness verification

Patent metadata

Field	Value
Publication number	US-12014571-B2
Application number	US-202117333915-A
Country	US
Kind code	B2
Filing date	May 28, 2021
Priority date	Feb 14, 2018
Publication date	Jun 18, 2024
Grant date	Jun 18, 2024

Abstract

Provided is a liveness verification method and device. A liveness verification device acquires a first image and a second image, and select one or more liveness models based on respective analyses of the first image and the second image, including analyses based on an object part being detected in the first image and/or the second image, and to verify, using the selected one or more liveness models, a liveness of the object based on the first image and/or the second image. The first image may be a color image and the second image may be an Infrared image.

First claim

What is claimed is:

1. A mobile device comprising:
one or more sensors configured to acquire a first image and a second image;
a memory configured to store a plurality of machine learning liveness detection models, wherein each of the plurality of machine learning liveness detection models are pretrained to determine whether a corresponding object, within a corresponding one or more images, is a live or non-live corresponding object; and
a processor configured to:
select multiple machine learning liveness detection models, from among the plurality of machine learning liveness detection models, based on respective analyses of the first image and/or the second image; and
determine, using the selected multiple machine learning liveness detection models, whether an object is a live object or a non-live object based on the first image and/or the second image.

2. The device of claim 1 , wherein, for the selecting of the multiple machine learning liveness detection models, the processor is configured to select a first machine learning liveness detection model and a second machine learning liveness detection model in response to respectively detecting, in the analyses, a respective target part or a respective object part in the first image and in the second image.

3. The device of claim 1 ,
wherein, for the selecting of the one or more machine learning liveness detection models, the processor is configured to select a part comparison machine learning model in response to a respective detection of a respective target part or a respective object part, which includes the respective target part and other parts, in the first image and in the second image, and
wherein, for the determination of whether the object is the live object or whether the object is the non-live object, the processor is configured to perform the determination of whether the object is the live object or whether the object is the non-live object using the part comparison machine learning model provided a first image patch of an eye of the first image and a second image patch of an eye of the second image.

4. The device of claim 3 , wherein the first image is a color image and the second image is an Infrared image.

5. The device of claim 3 , wherein each respective target part is an eye target part, and each respective object part is a face or portion of the face including facial parts in addition to the eye target part.

6. The device of claim 3 , wherein, for the selecting of the part comparison machine learning model, the processor is configured to select the part comparison model in response to the respective target part in the first image being determined located in a region of the first image.

7. The device of claim 1 , wherein the multiple machine learning liveness detection models include at least one second machine learning liveness detection model respectively configured to calculate at least one second liveness score in consideration of any one or any combination of any two or more of an object region patch of the second image, a first target region patch of the second image, and a second target region patch of the second image, and
the processor is further configured to perform the determining of whether the object is the live object or whether the object is the non-live object in further consideration of the at least one second liveness score.

8. The device of claim 1 , wherein the multiple machine liveness detection models include at least one first machine learning liveness detection model respectively configured to calculate at least one first liveness score in consideration of any one or any combination of any two or more of an entire region patch of the first image, an object region patch of the first image, and/or a determined region of interest (ROI) patch of the first image, and
the processor is further configured to perform the determining of whether the object is the live object or whether the object is the non-live object in further consideration of the at least one first liveness score.

9. The device of claim 1 ,
wherein the multiple machine learning liveness detection models include a first machine learning liveness detection model and a second machine learning liveness detection model,
wherein, for the determining of whether the object is the live object or whether the object is the non-live object, the processor is configured to determine a liveness of the object based on any one or any combination of a first liveness score and a second liveness score, and
wherein the first liveness score is calculated based on the first machine learning liveness detection model from a first image patch corresponding to an object part or a target part detected in the first image, and the second liveness score is calculated based on the second machine learning liveness detection model from a second image patch corresponding to the object part or the target part detected in the second image.

10. The device of claim 9 , wherein, for the determining of the liveness of the object, the processor is configured to determine the liveness of the object based on the first liveness score and the second liveness score.

11. The device of claim 1 ,
wherein, for the determining of whether the object is the live object or whether the object is the non-live object, the processor is configured to determine, based on a part comparison model, a liveness of the object from a first image patch corresponding to a target part and a second image patch corresponding to the target part, wherein the part comparison model is one of the one or more machine learning liveness detection models, and
wherein the first image patch and the second image patch respectively include different image modality information.

12. The device of claim 11 , wherein, with respect to the different image modality information, the first image patch includes visual spectrum image information and the second image patch includes non-visual spectrum image information, the first image patch includes color image information and the second image patch includes non-color image information, or the first image patch includes light spectrum information and the second image patch is a depth image patch.

13. The device of claim 11 , wherein, for the determining of the liveness of the object, the processor is further configured to:
extract a first part feature from the first image patch based on a feature extraction model of the part comparison model;
extract a second part feature from the second image patch based on the feature extraction model; and
determine whether the object is the live object or whether the object is the non-live object based on a difference between the extracted first part feature and the extracted second part feature.

14. The device of claim 11 , wherein, for the determining of whether the object is the live object or whether the object is the non-live object, the processor is configured to determine, based on the part comparison model, a liveness of the object from differential information between the first image patch and the second image patch.

15. The device of claim 1 , wherein, for the determining of whether the object is the live object or whether the object is the non-live object, the processor is configured to determine a liveness of the object based on a determined correlation level between a first image patch corresponding to a target part extracted from the first image and a second image patch determined in response to the first image patch.

16. The device of claim 1 , wherein the one or more sensors comprises:
a color image sensor configured to capt

Assignees

• Samsung Electronics Co Ltd

Inventors

• Park Sungun
• Yang Hyeongwook
• Sandhan Tushar Balasaheb
• Yu Jiaqian
• Xu Jingtao
• Kwak Youngjun
• YOO Juwoan
• Han Jae Joon

Classifications

• G06V10/803

  of input or preprocessed data · CPC title

• G06F18/253

  of extracted features · CPC title

• G06F18/251

  of input or preprocessed data · CPC title

• G06F18/211

  Selection of the most significant subset of features · CPC title

• G06V40/193

  Preprocessing; Feature extraction · CPC title