Method and apparatus with image fusion

Patent metadata

Field	Value
Publication number	US-12315115-B2
Application number	US-202217947366-A
Country	US
Kind code	B2
Filing date	Sep 19, 2022
Priority date	Feb 9, 2018
Publication date	May 27, 2025
Grant date	May 27, 2025

Abstract

Disclosed is an image fusion method and apparatus. The fusion method includes detecting first feature points of an object in a first image frame from the first image frame; transforming the first image frame based on the detected first feature points and predefined reference points to generate a transformed first image frame; detecting second feature points of the object in a second image frame from the second image frame; transforming the second image frame based on the detected second feature points and the predefined reference points to generate a transformed second image frame; and generating a combined image by combining the transformed first image frame and the transformed second image frame.

First claim

What is claimed is:

1. An image fusion method, comprising:
capturing a first facial image in a low-luminance environment;
extracting first feature points from the first facial image;
capturing a second facial image in the low-luminance environment;
extracting second feature points from the second facial image;
obtaining a combined image by combining the first facial image and the second facial image into a single image based on the first feature points and the second feature points, and
performing inverse transformation of the combined image,
wherein the obtaining of the combined image comprises warping the first facial image and the second facial image such that the first feature points and the second feature points are placed at positions of predefined reference points.

2. The image fusion method of claim 1 , wherein the obtaining of the combined image comprises combining the first facial image and the second facial image using any one or any combination of any two or more of a summing, an averaging, a weighted summing, or a weighted averaging of pixel values at corresponding positions in the first facial image and the second facial image.

3. The image fusion method of claim 1 ,
wherein the combined image is an image having an improved brightness over the first facial image and the second facial image.

4. The image fusion method of claim 1 ,
wherein the obtaining of the combined image comprises combining the warped first facial image and the warped second facial image.

5. The image fusion method of claim 1 ,
wherein the detecting of the first feature points comprises detecting facial landmarks from the first facial image, and
wherein the detecting of the second feature points comprises detecting facial landmarks from the second facial image.

6. The image fusion method of claim 1 , further comprising:
transforming the combined image based on a correspondence between the second feature points and the predefined reference points.

7. The image fusion method of claim 6 , further comprising:
extracting a feature for face verification from the transformed combined image.

8. The image fusion method of claim 7 , further comprising:
authenticating a user based on the extracted feature for face verification.

9. The image fusion method of claim 1 , further comprising:
measuring an image quality of the first image; and
determining to detect the first feature points from the first facial image, in response to the measured image quality of the first facial image satisfying a preset condition.

10. The image fusion method of claim 9 ,
wherein the measuring of the image quality comprises measuring a brightness of the first facial image, and
wherein the determining comprises determining to detect the first feature points from the first facial image, in response to the measured brightness of the first facial image being less than a preset threshold value.

11. An image fusion apparatus, comprising:
a processor configured to:
extract first feature points from a first facial image captured in a low-luminance environment;
extract second feature points from a second facial image captured in the low-luminance environment;
obtain a combined image by combining the first facial image and the second facial image into a single image based on the first feature points and the second feature points, and
performing inverse transformation of the combined image,
wherein, for the combination of the first facial image and the second facial image, the processor is configured to warp the first facial image and the second facial image such that the first feature points and the second feature points are placed at positions of predefined reference points.

12. The image fusion apparatus of claim 11 ,
wherein, for the obtainment of the combined image, the processor is configured to combine the first facial image and the second facial image using any one or any combination of any two or more of a summing, an averaging, a weighted summing, or a weighted averaging of pixel values at corresponding positions in the first facial image and the second facial image.

13. The image fusion apparatus of claim 11 ,
wherein the combined image is an image having an improved brightness over the first facial image and the second facial image.

14. The image fusion apparatus of claim 11 , wherein, for the obtainment of the combined image, the processor is further configured to combine the warped first facial image and the warped second facial image.

15. The image fusion apparatus of claim 11 ,
wherein the first feature points are facial landmarks detected from the first facial image, and
wherein the second feature points are facial landmarks detected from the second facial image.

16. The image fusion apparatus of claim 15 , wherein the processor is further configured to transform the combined image based on a correspondence between the second feature points and the predefined reference points.

17. The image fusion apparatus of claim 16 ,
wherein the processor is further configured to extract a feature for face verification from the transformed combined image.

18. The image fusion apparatus of claim 17 , wherein the processor is further configured to authenticate a user based on the extracted feature for face verification.

19. The image fusion apparatus of claim 11 , wherein the processor is further configured to:
measure an image quality of the first facial image; and
determine to detect the first feature points from the first facial image, in response to the measured image quality of the first facial image satisfying a preset condition.

20. An image fusion apparatus, comprising:
a camera configured to capture a first facial image and a second facial image in a low-luminance environment; and
a processor configured to:
extract first feature points from the first facial image;
extract second feature points from the second facial image;
obtain a combined image by combining the first facial image and the second facial image into a single image based on the first feature points and the second feature points, and
performing inverse transformation of the combined image, wherein, for the combination of the first facial image and the second facial image, the processor is configured to warp the first facial image and the second facial image such that the first feature points and the second feature points are placed at positions of predefined reference points.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Ko Minsu
• Han Seungju
• Han Jaejoon
• Kim Jihye
• Park Sungun
• Choi Chang Kyu

Classifications

• G06T2207/30168

  Image quality inspection · CPC title

• G06T7/0002

  Inspection of images, e.g. flaw detection · CPC title

• G06T3/153

  using elastic snapping · CPC title

• G06T3/18 Primary

  Image warping, e.g. rearranging pixels individually · CPC title

• G06T5/80

  Geometric correction · CPC title