Systems and methods for defective pixel correction with neighboring pixels

Patent metadata

Field	Value
Publication number	US-9131196-B2
Application number	US-201213724574-A
Country	US
Kind code	B2
Filing date	Dec 21, 2012
Priority date	May 31, 2012
Publication date	Sep 8, 2015
Grant date	Sep 8, 2015

Abstract

The present disclosure generally relates to systems and methods for image data processing. In certain embodiments, an image processing pipeline may detect and correct a defective pixel of image data acquired using an image sensor. The image processing pipeline may receive an input pixel of the image data acquired using the image sensor. The image processing pipeline may then identify a set of neighboring pixels having the same color component as the input pixel and remove two neighboring pixels from the set of neighboring pixels thereby generating a modified set of neighboring pixels. Here, the two neighboring pixels correspond to a maximum pixel value and a minimum pixel value of the set of neighboring pixels. The image processing pipeline may then determine a gradient for each neighboring pixel in the modified set of neighboring pixels and determine whether the input pixel includes a dynamic defect or a speckle based at least in part on the gradient for each neighboring pixel in the modified set of neighboring pixels.

First claim

What is claimed is:

1. An image signal processing system comprising:
an image sensor configured to receive image data;
an image processing pipeline configured to detect and correct a defective image pixel by:
receiving an input pixel of the image data acquired using the image sensor;

identifying a set of neighboring pixels having the same color component as the input pixel;
removing two neighboring pixels from the set of neighboring pixels thereby generating a modified set of neighboring pixels, wherein the two neighboring pixels correspond to a maximum pixel value and a minimum pixel value of the set of neighboring pixels;
determining a gradient for each neighboring pixel in the modified set of neighboring pixels; and
determining whether the input pixel comprises a speckle based at least in part on the gradient for each neighboring pixel in the modified set of neighboring pixels, wherein determining whether the input pixel comprises the speckle comprises:
determining an average pixel value based at least in part on each pixel value in the modified set of neighboring pixels;
determining an average gradient value based at least in part on a pixel value that corresponds to the input pixel and the average pixel value;
determining a despeckle threshold value by:
determining at least two brightness levels of a plurality of brightness levels, wherein the at least two brightness levels comprise a brightness level above and below a brightness value that corresponds to the average pixel value;
determining a first filter coefficient based at least in part on a first data array, the at least two brightness levels, at least two brightness values associated with the at least two brightness levels, and the average pixel value, wherein the first data array comprises a first plurality of threshold values indexed with respect to a color component of the input pixel and a brightness level of the plurality of brightness levels that corresponds to the average pixel value;
determining a second filter coefficient based at least in part on a second data array, the at least two brightness levels, the at least two brightness values associated with the at least two brightness levels, and the average pixel value, wherein the second data array comprises a second plurality of threshold values indexed with respect to the color component of the input pixel and the brightness level of the plurality of brightness levels that corresponds to the average pixel value; and
interpolating the first filter coefficient and the second filter coefficient;

comparing the average gradient value to the despeckle threshold value; and
identifying the input pixel as comprising the speckle if the average gradient value is greater than the despeckle threshold value.

2. The image signal processing system of claim 1 , wherein the image processing pipeline is configured to correct the defective image pixel by:
determining a replacement value for the input pixel if the input pixel comprises the speckle; and
replacing a current value of the input pixel with the replacement value if the input pixel comprises the speckle.

3. The image signal processing system of claim 1 , wherein the image processing pipeline is configured to identify the set of neighboring pixels by:
determining the color of the input pixel; and
identifying eight closest neighboring pixels of the same color in an imaging frame, wherein the eight neighboring pixels form a three-by-three pixel block.

4. The image signal processing system of claim 1 , wherein the gradient for each neighboring pixel in the modified set of neighboring pixels is a directional pixel-to-pixel gradient calculated with respect to the input pixel and each of the neighboring pixels in the modified set of neighboring pixels.

5. An image signal processing system comprising:
an image processing pipeline configured to detect and correct a defective image pixel by:
receiving an input pixel of image data acquired using an image sensor;
identifying a set of neighboring pixels having the same color component as the input pixel;
removing two neighboring pixels from the set of neighboring pixels thereby generating a modified set of neighboring pixels, wherein the two neighboring pixels correspond to a maximum pixel value and a minimum pixel value of the set of neighboring pixels;
determining a gradient for each neighboring pixel in the modified set of neighboring pixels; and
determining whether the input pixel comprises a dynamic defect based at least in part on the gradient for each neighboring pixel in the modified set of neighboring pixels, wherein determining whether the input pixel comprises the dynamic defect comprises:
comparing each gradient for each neighboring pixel in the modified set of neighboring pixels to an adaptive dynamic defect threshold value, wherein the adaptive dynamic defect threshold value is determined by:
determining a third filter coefficient based at least in part on a third data array, the average pixel value, the at least two brightness levels, and the at least two brightness values associated with the at least two brightness levels, wherein the third data array comprises a third plurality of threshold values indexed with respect to the color component of the input pixel and the brightness level of the plurality of brightness levels that corresponds to the average pixel value;
determining a fourth filter coefficient based at least in part on a fourth data array, the at least two brightness levels, the at least two brightness values associated with the at least two brightness levels, and the average pixel value, wherein the fourth data array comprises a fourth plurality of threshold values indexed with respect to the color component of the input pixel and the brightness level of the plurality of brightness levels that corresponds to the average pixel value; and
interpolating the third filter coefficient and the fourth filter coefficient;

counting a number of gradients that is less than or equal to the adaptive dynamic defect threshold value;
comparing the number to a maximum value; and
determining that the input pixel comprises the dynamic defect if the number is less than or equal to the maximum value.

6. The image signal processing system of claim 5 , wherein the maximum value is selected based at least in part on a location of the input pixel.

7. The image signal processing system of claim 5 , wherein the adaptive dynamic defect threshold value is determined based at least in part on an average pixel value of the modified set of neighboring pixels, each pixel value in the set of neighboring pixels, and at least two brightness levels in a plurality of brightness levels, wherein the at least two brightness levels comprise a brightness level above and below a brightness value that corresponds to the average pixel value.

8. The image signal processing system of claim 7 , wherein the plurality of brightness levels comprise five equally spaced brightness levels defined between 0 and 2^16.

9. The image signal processing system of claim 5 , wherein the image processing pipeline is configured to correct the defective image pixel when the input pixel comprises the dynamic defect, wherein the input pixel is corrected based at least in part on a plurality of pixel values that correspond to a 5×5 array of pixels that surround the input pixel.

10. The image signal processing system of claim 9 , wherein the image processing pipeline is configured to correct the defective image pixel by convolving the 5×5 array of pixels with a symmetric filter that has a 5×5 spatial support.

11. The image signal processing system of claim 10 , wherein the 5×5 spatial support compris

Assignees

• Apple Inc

Inventors

• Lim Suk Hwan
• Mills Christopher L
• Lin Sheng

Classifications

• H04N5/142 Primary

  Edging; Contouring · CPC title

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• H04N23/6811

  based on the image signal · CPC title

• H04N23/88

  for colour balance, e.g. white-balance circuits or colour temperature control · CPC title

• H04N25/611

  Correction of chromatic aberration · CPC title