Apparatus and method with image processing

Patent metadata

Field	Value
Publication number	US-12469103-B2
Application number	US-202217903480-A
Country	US
Kind code	B2
Filing date	Sep 6, 2022
Priority date	Dec 27, 2021
Publication date	Nov 11, 2025
Grant date	Nov 11, 2025

Abstract

An apparatus with image processing includes: one or more processors configured to: generate a full sampling image by demosaicing a first color filter array (CFA)-based input image; based on a trained color conversion model trained for color conversion and noise suppression of the full sampling image, determine a bias and a conversion matrix corresponding to each pixel of the full sampling image; and based on the conversion matrix and the bias, convert a color space of the full sample image, corresponding to the first CFA, into a color space corresponding to a second CFA.

First claim

What is claimed is:

1 . An apparatus with image processing, the apparatus comprising:
one or more processors configured to:
generate a full sampling image by demosaicing a first color filter array (CFA)-based input image;
based on a trained color conversion model trained for color conversion and noise suppression of the full sampling image, determine a bias and a conversion matrix corresponding to each pixel of the full sampling image; and
based on the conversion matrix and the bias, convert a color space of the full sample image, corresponding to the first CFA, into a color space corresponding to a second CFA.

2 . The apparatus of claim 1 , wherein the trained color conversion model is trained based on a loss function corresponding to a difference between another full sampling image, determined to correspond to a training image, of a color space corresponding to a second CFA and ground truth data corresponding to the training image.

3 . The apparatus of claim 1 , wherein the trained color conversion model comprises a first output layer configured to output the conversion matrix corresponding to the full sampling image and a second output layer configured to output the bias corresponding to the full sampling image.

4 . The apparatus of claim 1 , wherein the trained color conversion model is configured to control, based on a noise parameter corresponding to brightness of the CFA-based input image, a level of noise reduction of an image output using the trained color conversion model.

5 . The apparatus of claim 1 , wherein, for the generating of the full sampling image, the one or more processors are configured to:
based on a determined weight corresponding to each channel of the first CFA and a value of a channel of the first CFA corresponding to each pixel of an input image, generate the full sampling image comprising values of channels of the first CFA corresponding to each of the pixels of the input image; and update values of channels corresponding to each pixel of the full sampling image by applying the full sampling image to at least one trained layer.

6 . The apparatus of claim 5 , wherein
the determined weight corresponding to each channel of the first CFA is determined using a trained demosaicing model, the at least one trained layer is comprised in the trained demosaicing model, and the trained demosaicing model is trained based on a loss function based on a difference between the full sampling image of the color space corresponding to the first CFA determined to correspond to a training image and ground truth data corresponding to the training image.

7 . The apparatus of claim 1 , wherein, for the converting of the color space of the full sampling image, the one or more processors are configured to, based on a preset equation, by performing an operation with the conversion matrix, the bias, and a pixel value of the full sampling image, convert values of channels of the first CFA corresponding to each pixel of the full sampling image into values of channels of the second CFA.

8 . The apparatus of claim 1 , wherein the one or more processors are configured to generate a second CFA-based image by mosaicing the converted full sampling image.

9 . The apparatus of claim 1 , wherein
each pixel of the first CFA-based input image comprises a value of any of channels comprised in the first CFA, each pixel of the full sampling image comprises values of the channels comprised in the first CFA, and each pixel of the converted full sampling image comprises values of channels comprised in the second CFA.

10 . The apparatus of claim 1 , wherein a color spectrum corresponding to the first CFA is wider than a color spectrum corresponding to the second CFA.

11 . The apparatus of claim 1 , wherein
the first CFA comprises complementary color filters in cyan, magenta, and yellow, and the second CFA comprises primary color filters in red, green, and blue.

12 . A processor-implemented method with image processing, the method comprising:
generating a full sampling image by demosaicing a first color filter array (CFA)-based input image; based on a trained color conversion model trained for color conversion and noise suppression of the full sampling image, determining a bias and a conversion matrix corresponding to each pixel of the full sampling image; and based on the conversion matrix and the bias, converting a color space of the full sampling image, corresponding to the first CFA, into a color space corresponding to a second CFA.

13 . The method of claim 12 , wherein the trained color conversion model is trained based on a loss function corresponding to a difference between another full sampling image, determined to correspond to a training image, of a color space corresponding to a second CFA and ground truth data corresponding to the training image.

14 . The method of claim 12 , wherein the trained color conversion model comprises a first output layer configured to output the conversion matrix corresponding to the full sampling image and a second output layer configured to output the bias corresponding to the full sampling image.

15 . The method of claim 12 , wherein the trained color conversion model is configured to control, based on a noise parameter corresponding to brightness of the CFA-based input image, a level of noise reduction of an image output using the trained color conversion model.

16 . The method of claim 12 , wherein the generating of the full sampling image comprises:
based on a determined weight corresponding to each channel of the first CFA and a value of channel of the first CFA corresponding to each pixel of the input image, generating the full sampling image comprising values of channels of the first CFA corresponding to each of the pixels of the input image; and updating values of channels corresponding to each pixel of the full sampling image by applying the full sampling image to at least one trained layer.

17 . The method of claim 16 , wherein
the determined weight corresponding to each channel of the first CFA is determined using a trained demosaicing model, the at least one trained layer is comprised in the trained demosaicing model, and the trained demosaicing model is trained based on a loss function based on a difference between the full sampling image of the color space corresponding to the first CFA determined to correspond to a training image and ground truth data corresponding to the training image.

18 . The method of claim 12 , wherein the converting of the color space of the full sampling image comprises, based on a preset equation, by performing an operation with the conversion matrix, the bias, and a pixel value of the full sampling image, converting values of channels of the first CFA corresponding to each pixel of the full sampling image into values of channels of the second CFA.

19 . The method of claim 12 , further comprising generating a second CFA-based image by mosaicing the converted full sampling image.

20 . A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 12 .

21 . An apparatus with image processing, the apparatus comprising:
one or more processors configured to:
generate a first full sampling image by demosaicing a training image obtained using a first color filter array (CFA);
generate, using a color conversion model, a second full sampling image, corre

Assignees

• Samsung Electronics Co Ltd

Inventors

• Cho Sung Kwang
• Kim Geonwoo
• Cho Yang Ho
• Kang Deokyoung
• Nam Dong Kyung

Classifications

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/20224

  Image subtraction · CPC title

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T5/70

  Denoising; Smoothing · CPC title