Method and apparatus with color conversion

Patent metadata

Field	Value
Publication number	US-12015883-B2
Application number	US-202217826906-A
Country	US
Kind code	B2
Filing date	May 27, 2022
Priority date	Nov 18, 2021
Publication date	Jun 18, 2024
Grant date	Jun 18, 2024

Abstract

An electronic device includes an image sensor including a color filter having a plurality of color channels, a memory storing a plurality of color conversion matrices and instructions, and a processor. The processor is configured to obtain noise information of a color image captured by the image sensor, select a target matrix from among the plurality of the color conversion matrices based on the obtained noise information, and generate a color converted image by applying the selected target matrix to the color image.

First claim

What is claimed is:

1. An electronic device comprising:
an image sensor comprising a color filter having a plurality of color channels;
a memory storing a plurality of color conversion matrices and instructions; and
a processor configured to:
obtain noise information of a color image captured by the image sensor;
select a target matrix from among the plurality of the color conversion matrices based on whether the obtained noise information meets a threshold; and
generate a color converted image by applying the selected target matrix to the color image,

wherein the noise information comprises any one or any combination of any two or more of a noise scale representing variation between raw color values of pixels in the color image, an exposure time corresponding to the color image, a focal number (f-number) of a camera including the image sensor, an International Organization for Standardization (ISO) value of the camera, and a white balance gain value applied to the color data used to generate the color image.

2. The electronic device of claim 1 , wherein a total number of the plurality of the color channels is four or more.

3. The electronic device of claim 1 , wherein the memory
stores a first color conversion matrix, as a color conversion matrix of at least one of plural candidate color conversion matrices, having a minimum color conversion color difference, and
stores a second color conversion matrix, as another color conversion matrix among the plural candidate color conversion matrices, having a minimum noise amplification factor.

4. The electronic device of claim 3 , wherein, for the generation of the color converted image, the processor is further configured to:
in response to the noise scale of the color image being determined to be less than a threshold noise scale, apply the first color conversion matrix to the color image to generate the color converted image; and
in response to the noise scale of the color image being determined to be greater than or equal to the threshold noise scale, apply the second color conversion matrix to the color image to generate the color converted image.

5. The electronic device of claim 3 , wherein, for the generation of the color converted image, the processor is further configured to:
in response to the exposure time corresponding to the color image being determined to be greater than a threshold exposure time, apply the first color conversion matrix to the color image to generate the color converted image; and
in response to the exposure time corresponding to the color image being determined to be less than or equal to the threshold exposure time, apply the second color conversion matrix to the color image to generate the color converted image.

6. The electronic device of claim 3 , wherein, for the generation of the color converted image, the processor is further configured to:|
in response to the focal number (f-number) of the camera comprising the image sensor being less than a threshold f-number, apply the first color conversion matrix to the color image to generate the color converted image; and
in response to the f-number of the camera being greater than or equal to the threshold f-number, apply the second color conversion matrix to the color image to generate the color converted image.

7. The electronic device of claim 3 , wherein, for the generation of the color converted image, the processor is further configured to:
in response to the International Organization for Standardization (ISO) value of the camera comprising the image sensor being less than a threshold ISO value, apply the first color conversion matrix to the color image to generate the color converted image; and
in response to the ISO value of the camera being greater than or equal to the threshold ISO value, apply the second color conversion matrix to the color image to generate the color converted image.

8. The electronic device of claim 3 , wherein, for the generation of the color converted image, the processor is further configured to:
in response to the white balance gain value applied to the color image being less than a threshold white balance gain value, apply the first color conversion matrix to the color image to generate the color converted image; and
in response to the white balance gain value applied to the color image being greater than and equal to the threshold white balance gain value, apply the second color conversion matrix to the color image to generate the color converted image.

9. The electronic device of claim 3 , wherein the processor is further configured to:
select one or more color channel combinations by selecting respective portions of color channels from the plurality of color channels;
generate a candidate color conversion matrix for each of the color channel combinations; and
model one candidate color conversion matrix, of the plural candidate color conversion matrices, corresponding to one of the color channel combinations, to minimize a difference between a matrix calculated by multiplying a matrix comprising raw color values of reference pixels corresponding to color channels comprised in the one of the color channel combinations by a candidate color conversion matrix corresponding to the one of the color channel combinations and a matrix comprising standard color values of the reference pixels.

10. The electronic device of claim 1 , wherein the processor is further configured to:
calculate respective color conversion color differences and noise amplification factors for plural candidate color conversion matrices of the plurality of color conversion matrices generated from at least plural color channels among the plurality of the color channels.

11. The electronic device of claim 10 , wherein the processor is further configured to:
generate raw color values of reference pixels by demosaicing raw data obtained from the image sensor, calculate converted color values of the reference pixels by applying a candidate color conversion matrix, among the plurality of candidate color conversion matrices, to the raw color values of the reference pixels; and
calculate a color conversion color difference of the candidate color conversion matrix based on a difference between the converted color values calculated for the reference pixels and standard color values of the reference pixels.

12. The electronic device of claim 10 , wherein the processor is further configured to:
perform a singular value decomposition (SVD) on each of at least the plural candidate color conversion matrices, among the plural candidate color conversion matrices, and respectively extract a diagonal matrix of the plural candidate color conversion matrices; and
calculate respective noise amplification factors of the at least plural candidate color conversion matrices based on corresponding singular values in the respective extracted diagonal matrices of the at least plural candidate color conversion.

13. The electronic device of claim 12 , wherein, for the calculating of the respective noise amplification factors, the processor is further configured to:
calculate a noise amplification factor, among the respective noise amplification factors, based on a ratio of a minimum element value to a maximum element value among element values comprised in a corresponding extracted diagonal matrix among the respectively extracted diagonal matrices, and store the noise amplification factor in the memory.

14. A processing-implemented method performed by the electronic device, the method comprising:
generating noise information, by a processor component of the electronic device, of a color image captured by a

Assignees

• Samsung Electronics Co Ltd

Inventors

• Cho Yang Ho
• Kim Geonwoo
• Cho Sung Kwang
• Nam Dong Kyung

Classifications

• H04N23/85 Primary

  for matrixing · CPC title

• H04N23/88

  for colour balance, e.g. white-balance circuits or colour temperature control · CPC title

• H04N23/84

  for processing colour signals · CPC title

• H04N9/7908

  Suppression of interfering signals at the reproducing side, e.g. noise · CPC title

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title