Color filter spectrum optimization method and electronic apparatus performing the same

Patent metadata

Field	Value
Publication number	US-12273631-B2
Application number	US-202318129450-A
Country	US
Kind code	B2
Filing date	Mar 31, 2023
Priority date	Sep 22, 2022
Publication date	Apr 8, 2025
Grant date	Apr 8, 2025

Abstract

An optimization method includes determining first information related to a difference between the color filter spectrum and the color spectrum or a difference between an image transformed by the color transformation matrix and a ground truth (GT) image in the reset color space, determining second information representing smoothness of the color filter spectrum, third information representing a transmittance of the color filter spectrum, calculating a cost value based on the first information, the second information, and the third information, compare the calculated cost value and a threshold, and updating one of the filter spectrum information and the color transformation matrix in response to the calculated cost value being equal to or greater than the threshold.

First claim

What is claimed is:

1. An optimization method performed by an electronic apparatus, the optimization method comprising:
determining first information related to a) a difference between filter spectrum information representing a color filter spectrum and spectrum information representing a color spectrum in a preset color space or b) a difference between an image transformed by a color transformation matrix and a ground truth (GT) image in the preset color space;
determining second information representing smoothness of the color filter spectrum;
determining third information representing a transmittance of the color filter spectrum;
calculating a cost value based on the first information, the second information, and the third information;
comparing the calculated cost value and a threshold; and
in response to the calculated cost value being equal to or greater than the threshold, updating one of the filter spectrum information and the color transformation matrix.

2. The optimization method of claim 1 , wherein the determining of the first information comprises:
obtaining filter spectrum information to which color transformation is applied by applying the color transformation matrix to the filter spectrum information; and
determining that a magnitude of a difference between the filter spectrum information to which the color transformation is applied and the spectrum information is the first information.

3. The optimization method of claim 1 , further comprising:
determining fourth information related to noise amplification of the color transformation matrix,
wherein the calculating of the cost value comprises calculating that a function value of an objective function based on a result of summing negative information of the third information, the first information, the second information, and the fourth information.

4. The optimization method of claim 3 , further comprising:
applying a first weight to the first information, a second weight to the second information, a third weight to the third information, and fourth weight to the fourth information.

5. The optimization method of claim 1 , wherein the determining of the first information comprises:
applying noise to an image comprising a plurality of spectral bands;
filtering the image to which the noise is applied based on the color filter spectrum;
performing image processing on the filtered image and obtaining a result image generated through the image processing;
performing first color transformation on the obtained result image based on the color transformation matrix and obtaining a first color transformed image corresponding to the transformed image;
applying the spectrum information to the image comprising the spectral bands and obtaining the GT image; and
determining that a magnitude of a difference between the obtained first color transformed image and the obtained GT image is the first information.

6. The optimization method of claim 5 , further comprising:
performing second color transformation on the obtained first color transformed image and obtaining a second color transformed image;
performing the second color transformation on the GT image and obtaining a color transformed GT image; and
determining that a magnitude of a difference between the obtained second color transformed image and the color transformed GT image is fifth information.

7. The optimization method of claim 6 , wherein the calculating of the cost value comprises calculating that a function value of an objective function based on a result of summing negative information of the third information, the first information, the second information, and the fifth information is the cost value.

8. The optimization method of claim 7 , further comprising:
applying a first weight to the first information, a second weight to the second information, a third weight to the third information, and a fifth weight to the fifth information.

9. The optimization method of claim 5 , wherein the image comprising the spectral bands comprises a hyperspectral image (HSI).

10. The optimization method of claim 1 , further comprising:
in response to the calculated cost value being less than the threshold, determining that the filter spectrum information is optimal filter spectrum information and that the color transformation matrix is an optimal color transformation matrix.

11. The optimization method of claim 1 , further comprising:
in response to the calculated cost value being less than the threshold, determining that the filter spectrum information and the color transformation matrix are a first optimal candidate pair;
determining a second optimal candidate pair by updating one of the filter spectrum information and the color transformation matrix and calculating a second cost value based on the determined second optimal candidate pair;
determining a third optimal candidate pair by updating another one of the filter spectrum information and the color transformation matrix and calculating a third cost value based on the third optimal candidate pair;
in response to a predetermined number of optimal candidate pairs comprising the first through third optimal candidate pairs being determined, selecting one from among the first, second and third optimal candidate pairs based on the first, second and third cost values calculated respectively for the determined optimal candidate pairs; and
determining that filter spectrum information corresponding to the selected optimal candidate pair is optimal filter spectrum information and that a color transformation matrix corresponding to the selected optimal candidate pair is an optimal color transformation matrix.

12. The optimization method of claim 1 , wherein
the determining of the second information comprises calculating a gradient of the filter spectrum information and determining a magnitude of the calculated gradient is the second information, and
the determining of the third information comprises determining that a magnitude of the filter spectrum information is the third information.

13. The optimization method of claim 1 , wherein
the filter spectrum information is in a matrix form, and
columns of the filter spectrum information in the matrix form represent a plurality of color channels, and rows of the filter spectrum information in the matrix form represent wavelength values in a visible band.

14. An electronic apparatus comprising:
a memory configured to store one or more instructions; and
a processor configured to execute the instructions,
wherein when the instructions are executed, the processor is configured to:
determine first information related to a a) difference between filter spectrum information representing a color filter spectrum and spectrum information representing a color spectrum in a present color space or b) a difference between an image transformed by a color transformation matrix and a ground truth (GT) image in the preset color space, determine second information representing smoothness of the color filter spectrum, determine third information representing a transmittance of the color filter spectrum, calculate a cost value based on the first information, the second information, and the third information, compare the calculated cost value and a threshold, and in response to the calculated cost value being equal to or greater than the threshold, update one of the filter spectrum information and the color transformation matrix.

15. The electronic apparatus of claim 14 , wherein the processor is further configured to:
obtain filter spectrum information to which color transformation is applied by app

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Geonwoo
• Cho Sung Kwang
• Cho Yang Ho
• Nam Dong Kyung

Classifications

• H04N25/134

  based on three different wavelength filter elements · CPC title

• H04N23/617

  Upgrading or updating of programs or applications for camera control · CPC title

• H04N1/603

  controlled by characteristics of the picture signal generator or the picture reproducer · CPC title

• H04N23/85 Primary

  for matrixing · CPC title

• H04N23/84 Primary

  for processing colour signals · CPC title