Wide dynamic range image processing method and image signal processor using the same

Patent metadata

Field	Value
Publication number	US-9241127-B2
Application number	US-201414200171-A
Country	US
Kind code	B2
Filing date	Mar 7, 2014
Priority date	Mar 14, 2013
Publication date	Jan 19, 2016
Grant date	Jan 19, 2016

Abstract

An image processing method includes the operations of receiving N×N raw data from an N×N sub pixel array including a plurality of first exposure pixels and second exposure pixels having different exposure times, respectively; obtaining a difference value between a value obtained by normalizing an interpolated value of the first exposure pixels and an interpolated value of the second exposure pixels for each channel in the N×N sub pixel array based on the N×N raw data; setting a mismatching value to a maximum value among difference values obtained for the respective channels; and setting a corrected value of a central pixel in the N×N sub pixel array based on the N×N raw data and based on the mismatching value and a threshold value.

First claim

What is claimed is:

1. An image signal processor comprising:
a receiver configured to receive N×N raw data from an N×N sub pixel array including a plurality of first exposure pixels and second exposure pixels having different exposure times and a plurality of channels;
a mismatching value setting block configured to obtain a difference value between a value obtained by normalizing an interpolated value of the first exposure pixels and an interpolated value of the second exposure pixels for each channel in the N×N sub pixel array based on the N×N raw data and to set a mismatching value to a maximum value among difference values obtained for the respective channels;
a corrected value setting block configured to set a corrected value of a central pixel in the N×N sub pixel array based on the N×N raw data and based on the mismatching value and a threshold value;
a corrected data output unit configured to sequentially receive corrected values of pixels from the corrected value setting block and to generate and output corrected data by updating pixel values of pixels with the corrected values;
a first interpolated value setting block configured to set a first interpolated value to a value obtained by normalizing an interpolated value of the first exposure pixels having the same channel as the central pixel in the N×N sub pixel array based on the N×N raw data;
a second interpolated value setting block configured to set a second interpolated value to a value obtained by normalizing an interpolated value of the second exposure pixels having the same channel as the central pixel in the N×N sub pixel array based on the N×N raw data; and
a central value setting block configured to set a central value to a value obtained by normalizing a pixel value of the central pixel when the central pixel is one of the plurality of first exposure pixels and to the pixel value of the central pixel when the central pixel is one of the plurality of second exposure pixels based on the N×N raw data,
wherein the corrected value setting block is configured to set the corrected value based on the first interpolated value, the second interpolated value, and the central value when the mismatching value is less than the threshold value and based on the first interpolated value and the second interpolated value when the mismatching value is equal to or greater than the threshold value.

2. The image signal processor of claim 1 , further comprising:
a maximum average setting block configured to obtain a maximum value among pixel values in each of a plurality of adjacent areas of the central pixel based on the N×N raw data and to set a maximum average to a weighted average of the maximum values of the respective adjacent areas,
wherein the corrected value setting block is configured to set the corrected value through interpolation in which the first interpolated value, the second interpolated value, and the central value are given a different weight according to whether the mismatching value is less than the threshold value and a level of the maximum average.

3. The image signal processor of claim 2 , wherein
the corrected value setting block is configured to set first and second highs and first and second lows based on a ratio between an exposure time of the first exposure pixels and an exposure time of the second exposure pixels; and
the corrected value setting block is configured to set the corrected value such that when the mismatching value is less than the threshold value, the corrected value is the second interpolated value when the maximum average is less than the second low, is an interpolation value between the second interpolated value and the central value when the maximum average is greater than the second low and less than the second high, is the central value when the maximum average is greater than the second high and less than the first low, and is an interpolation value between the central value and the first interpolated value when the maximum average is greater than the first low and less than the first high, and is the first interpolated value when the maximum average is greater than the first high.

4. The image signal processor of claim 3 , wherein
the corrected value setting block is configured to set a third low and a third high based on a ratio between an exposure time of the first exposure pixels and an exposure time of the second exposure pixels; and
the corrected value setting block is configured to set the corrected value such that when the mismatching value is equal to or greater than the threshold value, the corrected value is the second interpolated value when the maximum average is less than the third low, is an interpolation value between the second interpolated value and the first interpolated value when the maximum average is greater than the third low and less than the third high, and is the first interpolated value when the corrected value is greater than the third high.

5. The image signal processor of claim 1 , wherein the value obtained by normalizing the pixel value of the central pixel is a value obtained by multiplying the pixel value of the central pixel by a second exposure time and dividing a multiplication result by a first exposure time.

6. A system comprising:
an image sensor configured to generate N×N raw data as pixel values from an N×N sub pixel array having a plurality of pixels, the plurality of pixels associated with one of a plurality of colors and being one of a first exposure type and a second exposure type, a first exposure time of the first exposure type being less than a second exposure time of the second exposure type; and
an image signal processor configured to receive the pixel values, generate a difference value based on interpolated values of pixels of different exposure types and associated with the same color, adjust a selected pixel value of the pixel values based on a first process or a second process, the image signal processor configured to select the first process or the second process based on the difference value, the selected pixel value associated with a selected pixel of the plurality of pixels, the image signal processor including,
a first interpolated value setting block configured to set a first interpolated value to a value obtained by normalizing an interpolated value of the first exposure type pixels having a same channel as a central pixel in the N×N sub pixel array based on the pixel values, the central pixel being the selected pixel,
a second interpolated value setting block configured to set a second interpolated value to a value obtained by normalizing an interpolated value of the second exposure types pixels having the same channel as the central pixel in the N×N sub pixel array based on the pixel values, and
a central value setting block configured to set a central value to a value obtained by normalizing a pixel value of the central pixel when the central pixel is one of the plurality of first exposure types pixels and to the pixel value of the central pixel when the central pixel is one of the plurality of second exposure type pixels based on the pixel values, and
a corrected value setting block is configured to set a corrected value of the central pixel based on the first interpolated value, the second interpolated value, and the central value when a mismatching value is less than a threshold value and based on the first interpolated value and the second interpolated value when the mismatching value is equal to or greater than the threshold value.

7. The system of claim 6 , wherein the image signal processor is configured to generate a maximum average based on maximum values among the pixels values in respective areas of the image sensor, the first process and the second process being based on the maximum average.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Sfaradi Roee
• Eisenberg Amit
• Monzon Oded

Classifications

• H04N25/134

  based on three different wavelength filter elements · CPC title

• H04N23/73

  by influencing the exposure time · CPC title

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• H04N25/583

  with different integration times · CPC title