Apparatus and method for high dynamic range imaging using spatially varying exposures

Patent metadata

Field	Value
Publication number	US-9363447-B2
Application number	US-201414581853-A
Country	US
Kind code	B2
Filing date	Dec 23, 2014
Priority date	Jun 4, 1999
Publication date	Jun 7, 2016
Grant date	Jun 7, 2016

Abstract

Apparatus and methods are provided for obtaining high dynamic range images using a low dynamic range image sensor. The scene is exposed to the image sensor in a spatially varying manner. A variable-transmittance mask, which is interposed between the scene and the image sensor, imposes a spatially varying attenuation on the scene light incident on the image sensor. The mask includes light transmitting cells whose transmittance is controlled by application of suitable control signals. The mask is configured to generate a spatially varying light attenuation pattern across the image sensor. The image frame sensed by the image sensor is normalized with respect to the spatially varying light attenuation pattern. The normalized image data can be interpolated to account for image sensor pixels that are either under or over exposed to enhance the dynamic range of the image sensor.

First claim

The invention claimed is:

1. A system for high dynamic range imaging, the system comprising:
an image sensor comprising an array of light-sensing elements, wherein the image sensor detects an image of a scene and provides pixel values that represent light intensities impinging on respective ones of the light-sensing elements;
a mask comprising a plurality of light-transmitting cells that are each configured to attenuate the intensity of light transmitted through the mask onto the light-sensing elements such that the scene is exposed to the light-sensing elements in a spatially varying manner with each of the light-sensing elements having a corresponding exposure value indicative of the transparency of one or more light-transmitting cells through which light impinging on the light-sensing element passes, wherein the mask is interposed between the scene and the array of light-sensing elements of the image sensor; and
an image processor coupled to the image sensor for receiving the pixel values provided thereby, wherein the image processor determines high dynamic range image data using at least the pixel values as function of the exposure values.

2. The system of claim 1 , wherein the image sensor provides a saturated pixel value when the light intensity impinging on one of the light-sensing elements is greater than a first threshold level and provides a blackened pixel value when the light intensity impinging on one of the light-sensing elements is less than a second threshold level.

3. The system of claim 1 , wherein the image sensor has a low dynamic range relative to the range of intensities of the image of the scene.

4. The system of claim 1 , wherein the mask is arranged with respect to the image sensor such that each light-sensing element of the array of light-sensing elements corresponds to a light-transmitting cell.

5. The system of claim 1 , wherein the plurality of light-transmitting cells have transparencies that form a regular pattern of across the mask.

6. The system of claim 5 , wherein the pattern is comprised of repeating groups of four neighboring light-transmitting cells that each form a similar two by two pattern of transparencies.

7. The system of claim 1 , wherein the image processor further comprises a normalizer for mapping the pixel values by a function of the exposure values to derive normalized pixel values.

8. The system of claim 7 , wherein the image processor divides each of the pixel values with the exposure value corresponding to the light-sensing element receiving light intensity represented by the pixel value to derive the normalized pixel values.

9. The system of claim 1 , further comprising a memory for receiving and storing at least one of the pixel values and the exposure values.

10. A method for high dynamic range imaging, the method comprising:
exposing an image sensor to light from a scene to sense an image frame, wherein the image sensor comprises an array of light-sensing elements,
wherein the light from the scene is passed through a mask that is interposed between the scene and the array of light-sensing elements of the image sensor, wherein the mask comprises a plurality of light-transmitting cells that are each configured to attenuate the intensity of light transmitted through the mask onto the light-sensing elements such that the scene is exposed to the light-sensing elements in a spatially varying manner with each of the light-sensing elements having a corresponding exposure value indicative of the transparency of one or more light-transmitting cells through which light impinging on the light-sensing element passes; and
determining high dynamic range imaging data based at least in part on the pixel values as a function of the exposure values.

11. The method of claim 10 , wherein the image sensor provides a saturated pixel value when the light intensity impinging on one of the light-sensing elements is greater than a first threshold level and provides a blackened pixel value when the light intensity impinging on one of the light sensing elements is below a second threshold level.

12. The method of claim 10 , wherein the image sensor has a low dynamic range relative to the range of intensities of the image of the scene.

13. The method of claim 10 , wherein the mask is arranged with respect to the image sensor such that each light-sensing element of the array of light-sensing elements corresponds to a light-transmitting cell.

14. The method of claim 10 , wherein the plurality of light-transmitting cells have transparencies that form a regular pattern of across the mask.

15. The method of claim 14 , wherein the pattern is comprised of repeating groups of four neighboring light-transmitting cells that each form a similar two by two pattern of transparencies.

16. The method of claim 10 , further comprising normalizing the pixel values, wherein the pixel values are mapped as a function of the exposure values to derive the normalized pixel values.

17. The method of claim 16 , further comprising dividing each of the pixel values with the exposure value corresponding to the light-sensing element receiving light intensity represented by the pixel value to derive the normalized pixel values.

18. The method of claim 16 , further comprising calibrating the pixel values provided by the image sensor according to a response function of the image sensor to derive linear response pixel values prior to normalizing the pixel values.

Assignees

• Univ Columbia
• Sony Corp

Inventors

• Nayar Shree K
• Mitsunaga Tomoo

Classifications

• H04N23/741 Primary

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N23/70 Primary

  Circuitry for compensating brightness variation in the scene · CPC title

• H04N5/2355 Primary

  Electricity · mapped topic

• H04N5/235

  Electricity · mapped topic