Method and apparatus for image flare mitigation

Patent metadata

Field	Value
Publication number	US-9497427-B2
Application number	US-201313942201-A
Country	US
Kind code	B2
Filing date	Jul 15, 2013
Priority date	Jul 15, 2013
Publication date	Nov 15, 2016
Grant date	Nov 15, 2016

Abstract

An imaging system may include a camera module with an image sensor having an array of image sensor pixels and one or more lenses that focus light onto the array of image sensor pixels. The array of image sensor pixels may include a corresponding array of color filter elements. The system may include circuitry configured to detect and mitigate flare artifacts in image data captured using the image sensor. Detecting the flare artifacts may include detecting color mismatches in the captured image data and performing edge-detection operations to determine whether the color mismatches are in an edge region of the image data. If the color mismatches are detected in an edge region, no flare-mitigation operations may be performed. If the color mismatches are in a flat region of the captured image, flare-mitigation operations may be performed.

First claim

What is claimed is:

1. An imaging system, comprising:
an image sensor comprising a plurality of image pixels, wherein the plurality of image pixels are configured to capture image data;
at least one lens that focuses image light onto the image sensor; and
processing circuitry configured to detect and mitigate flare artifacts in a frame of the captured image data by detecting color mismatches in portions of the frame of the captured image data captured by at least first and second image pixels of the plurality of image pixels, wherein the processing circuitry detects color mismatches in the portions of the frame of the captured image data that is output from the first and second image pixels by comparing a ratio between a first portion of the frame of the captured image data captured by the first image pixel and a second portion of the frame of the captured image data captured by the second image pixel to a predetermined threshold.

2. The imaging system defined in claim 1 wherein the processing circuitry is further configured to detect the flare artifacts by performing edge-detection operations on the frame of the captured image data.

3. The imaging system defined in claim 2 wherein image sensor comprises an array of image pixels and a corresponding array of color filter elements.

4. The imaging system defined in claim 3 wherein the corresponding array of color filter elements comprises repeating groups of red color filter elements, blue color filter elements, and green color filter elements.

5. The imaging system defined in claim 3 wherein the corresponding array of color filter elements comprises repeating groups of red color filter elements, blue color filter elements, and clear color filter elements.

6. The imaging system defined in claim 2 wherein image sensor comprises a plurality of arrays of image pixels and wherein each array of image pixels has a corresponding array of color filter elements having a common color.

7. The imaging system defined in claim 1 , wherein the first image pixel is adjacent to the second image pixel.

8. A method of generating images, comprising:
with an image sensor comprising a plurality of image pixels, capturing an image having image data;
with processing circuitry, detecting color mismatches in portions of the captured image data that is output from at least first and second clear image pixels of the plurality of image pixels; and
with the processing circuitry, detecting flare artifacts in at least some of the portions of the captured image data based on the detected color mismatches by comparing a difference between an image signal from the first clear image pixel and an additional image signal from the second clear image pixel to a predetermined threshold.

9. The method defined in claim 8 , further comprising:
with the processing circuitry, performing edge-detection operations on the captured image data.

10. The method defined in claim 9 wherein performing the edge-detection operations on the captured image data comprises performing the edge-detection operations on the captured image data prior to detecting the color mismatches in the portions of the captured image data.

11. The method defined in claim 9 wherein performing the edge-detection operations on the captured image data comprises performing the edge-detection operations on the captured image data after detecting the color mismatches in the portions of the captured image data.

12. The method defined in claim 9 , further comprising:
with the processing circuitry, performing flare-mitigation operations on the at least some of the portions of the captured image data.

13. The method defined in claim 12 wherein performing the flare-mitigation operations on the at least some of the portions of the captured image data comprises generating flare-indicating metadata.

14. The method defined in claim 12 wherein performing the flare-mitigation operations on the at least some of the portions of the captured image data comprises desaturating the at least some of the portions of the captured image data.

15. The method defined in claim 12 wherein performing the flare-mitigation operations on the at least some of the portions of the captured image data comprises modifying the at least some of the portions of the captured image data using a flare-specific color correction matrix.

16. The method defined in claim 8 , further comprising:
computing a ratio of the image signal and the additional image signal; and
comparing the computed ratio to an additional threshold.

17. A system, comprising:
a central processing unit;
memory;
input-output circuitry; and
an imaging device, wherein the imaging device comprises:
an image sensor having an array of image sensor pixels arranged in repeating unit cells each having a red pixel, a blue pixel, and two clear pixels;
at least one lens that focuses image light onto the image sensor; and
processing circuitry configured to detect flare artifacts in an image having image data captured using the image sensor by detecting color mismatches in a portion of the captured image data of the image captured by at least first and second image sensor pixels, wherein the processing circuitry is further configured to detect edges of objects that are being imaged in the portion of the captured image data, wherein the processing circuitry is further configured to perform flare mitigation operations when no object edges are detected in the portion of the captured image data, and wherein the processing circuitry is further configured to omit flare-mitigation operations on the captured image data when object edges are detected in the portion of the captured image data.

18. The system defined in claim 17 wherein the processing circuitry is further configured to perform flare-mitigation operations on the detected flare artifacts in response to determining that the portion of the captured image data is flat.

Assignees

• Semiconductor Components Ind Llc

Inventors

• Stern Jonathan Michael
• Black Robert A

Classifications

• H04N23/84 Primary

  for processing colour signals · CPC title

• H04N25/134

  based on three different wavelength filter elements · CPC title

• H04N23/81

  for suppressing or minimising disturbance in the image signal generation · CPC title

• H04N1/58 Primary

  Edge or detail enhancement; Noise or error suppression, e.g. colour misregistration correction (H04N1/62 takes precedence) · CPC title

• H04N5/23229

  Electricity · mapped topic