Bad pixel correction in image processing applications or other applications

Patent metadata

Field	Value
Publication number	US-2024202874-A1
Application number	US-202218068916-A
Country	US
Kind code	A1
Filing date	Dec 20, 2022
Priority date	Dec 20, 2022
Publication date	Jun 20, 2024
Grant date	—

Abstract

A method includes obtaining input image frames, including at least two captured using different capture conditions. The method also includes separating color channels of each image frame and generating at least one bad pixel map for each color channel of each image frame. Each bad pixel map is generated by identifying one or more outliers in pixel values in the color channel based on an intensity distribution of pixel values in an operation window within the color channel. The operation window has a window size based on the capture condition and/or local image content in the corresponding image frame. The method further includes combining the bad pixel maps and performing a morphological operation to refine the combined bad pixel map. In addition, the method includes using one or more coordinates of one or more bad pixels in the refined bad pixel map to update one or more pixel values of at least one image frame.

First claim

What is claimed is:

1 . A method comprising:
obtaining multiple input image frames, at least two of the input image frames captured using different capture conditions; separating color channels of each of the input image frames; generating at least one bad pixel map for each color channel of each input image frame, wherein each bad pixel map is generated by identifying one or more outliers in pixel values in the corresponding color channel based on an intensity distribution of the pixel values in a specified operation window within the corresponding color channel, the specified operation window having a window size based on at least one of: the capture condition associated with the corresponding image frame and local image content in the corresponding image frame; combining the bad pixel maps associated with the color channels to produce at least one combined bad pixel map; performing a morphological operation to refine the at least one combined bad pixel map and produce at least one refined bad pixel map; and using one or more coordinates of one or more bad pixels identified in the at least one refined bad pixel map to update one or more pixel values of at least one of the input image frames.

2 . The method of claim 1 , wherein the morphological operation comprises removing clusters of bad pixels identified in the at least one combined bad pixel map to produce the at least one refined bad pixel map.

3 . The method of claim 1 , wherein the morphological operation comprises spatial filtering of the at least one combined bad pixel map to produce the at least one refined bad pixel map.

4 . The method of claim 1 , wherein the morphological operation comprises performing erode operations and dilate operations, the erode operations replacing pixel values in the specified operation window with a minimum value, the dilate operations replacing the pixel values in the specified operation window with a maximum value.

5 . The method of claim 1 , wherein generating the at least one bad pixel map for each color channel of each input image frame comprises:
for each color channel of each input image frame:
identifying first and N th maximum and minimum pixel values in the color channel of the image frame, the first and N th maximum and minimum pixel values based on pixel values within the specified operation window;
identifying adaptive thresholds based on differences between the first and N th maximum and minimum pixel values; and
applying the adaptive thresholds to identify the one or more outliers in the pixel values of the color channel.

6 . The method of claim 5 , wherein identifying the adaptive thresholds comprises:
linearly updating maximum and minimum thresholds using predefined slopes until the maximum and minimum pixel values in the color channel of the image frame reach constant values.

7 . The method of claim 1 , wherein the different capture conditions comprise different exposure settings.

8 . An electronic device comprising:
at least one imaging sensor configured to capture multiple input image frames such that at least two of the input image frames are captured using different capture conditions; and at least one processing device configured to:
separate color channels of each of the input image frames;
generate at least one bad pixel map for each color channel of each input image frame, wherein, to generate each bad pixel map, the at least one processing device is configured to identify one or more outliers in pixel values in the corresponding color channel based on an intensity distribution of the pixel values in a specified operation window within the corresponding color channel, the specified operation window having a window size based on at least one of: the capture condition associated with the corresponding image frame and local image content in the corresponding image frame;
combine the bad pixel maps associated with the color channels to produce at least one combined bad pixel map;
perform a morphological operation to refine the at least one combined bad pixel map and produce at least one refined bad pixel map; and
use one or more coordinates of one or more bad pixels identified in the at least one refined bad pixel map to update one or more pixel values of at least one of the input image frames.

9 . The electronic device of claim 8 , wherein, to perform the morphological operation, the at least one processing device is configured to remove clusters of bad pixels identified in the at least one combined bad pixel map to produce the at least one refined bad pixel map.

10 . The electronic device of claim 8 , wherein, to perform the morphological operation, the at least one processing device is configured to spatially filter the at least one combined bad pixel map to produce the at least one refined bad pixel map.

11 . The electronic device of claim 8 , wherein:
to perform the morphological operation, the at least one processing device is configured to perform erode operations and dilate operations; to perform the erode operations, the at least one processing device is configured to replace pixel values in the specified operation window with a minimum value; and to perform the dilate operations, the at least one processing device is configured to replace the pixel values in the specified operation window with a maximum value.

12 . The electronic device of claim 8 , wherein, to generate the at least one bad pixel map for each color channel of each input image frame, the at least one processing device is configured to:
for each color channel of each input image frame:
identify first and N th maximum and minimum pixel values in the color channel of the image frame, the first and N th maximum and minimum pixel values based on pixel values within the specified operation window;
identify adaptive thresholds based on differences between the first and N th maximum and minimum pixel values; and
apply the adaptive thresholds to identify the one or more outliers in the pixel values of the color channel.

13 . The electronic device of claim 12 , wherein, to identify the adaptive thresholds, the at least one processing device is configured to linearly update maximum and minimum thresholds using predefined slopes until the maximum and minimum pixel values in the color channel of the image frame reach constant values.

14 . The electronic device of claim 8 , wherein the different capture conditions comprise different exposure settings.

15 . A non-transitory computer readable medium containing instructions that when executed cause at least one processor to:
obtain multiple input image frames, at least two of the input image frames captured using different capture conditions; separate color channels of each of the input image frames; generate at least one bad pixel map for each color channel of each input image frame, wherein the instructions that when executed cause the at least one processor to generate each bad pixel map comprise instructions that when executed cause the at least one processor to identify one or more outliers in pixel values in the corresponding color channel based on an intensity distribution of the pixel values in a specified operation window within the corresponding color channel, the specified operation window having a window size based on at least one of: the capture condition associated with the corresponding image frame and local image content in the corresponding image frame; combine the bad pixel maps associated with the color channels to produce at least one combined bad pixel map; perform a morphological operation to refine the a

Assignees

• Samsung Electronics Co Ltd

Inventors

• Suthar Madhuri
• Luu Tyler
• Le Nguyen Thang Long
• Glotzbach John W
• Sheikh Hamid R

Classifications

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T5/30 Primary

  Erosion or dilatation, e.g. thinning · CPC title

• G06T5/70

  Denoising; Smoothing · CPC title

• G06T3/4015 Primary

  Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns · CPC title

• G06T2207/20024

  Filtering details · CPC title