Empirical exposure normalization

Patent metadata

Field	Value
Publication number	US-11069040-B2
Application number	US-201916600999-A
Country	US
Kind code	B2
Filing date	Oct 14, 2019
Priority date	Oct 24, 2017
Publication date	Jul 20, 2021
Grant date	Jul 20, 2021

Abstract

An automated process empirically normalizes a “dark” image by adjusting the apparent exposure to compensate for nonlinearity in the luminance response of the image sensor. The process includes receiving at least two digital images, one of the digital images having an exposure value that is greater than that of another of the digital images. A reduced-resolution pair of images is produced from the at least two digital images. At least one representative scale factor is calculated from tonal values in the two images and at least one empirical scale factor is determined by selective interpolation between the representative scale factor and a comparative scale factor. The empirical scale factor is used in a function applied pixelwise to the darker of the digital images to produce an empirically normalized digital image.

First claim

What is claimed is:

1. A method comprising:
acquiring, by a processing device, a dark exposure of a first digital image and a light exposure of a second digital image, wherein the first digital image includes a first photographic content item and the second digital image includes a second photographic content item;
determining, by the processing device, that the first photographic content item matches the second photographic content item;
producing, by the processing device, a first reduced-resolution image from the first digital image and a second reduced-resolution image from the second digital image;
calculating, by the processing device, a light-tone representative scale factor and a dark-tone representative scale factor for each of the first reduced-resolution image and the second reduced-resolution image;
producing, by the processing device, a dark-tone empirical scale factor by selectively interpolating between the dark-tone representative scale factor and the light-tone representative scale factor for each of the first reduced-resolution image and the second reduced-resolution image; and
generating, by the processing device, an empirically normalized digital image by applying, pixelwise, a function including at least the dark-tone empirical scale factor to the first digital image or the second digital image.

2. The method of claim 1 , wherein the calculating of the dark-tone representative scale factor for each of the first reduced-resolution image and the second reduced-resolution image further comprises gathering a dark-tone pairwise set of scale factors from the first reduced-resolution image and the second reduced-resolution image.

3. The method of claim 1 , wherein selectively interpolating comprises interpolating among the light-tone representative scale factor, an updated value of the light-tone representative scale factor, and the dark-tone representative scale factor to update a value of the dark-tone empirical scale factor.

4. The method of claim 3 , further comprising interpolating a normal scale factor based at least in part on exchangeable image file data.

5. The method of claim 1 , wherein the function includes a camera sensor parameter indicating a luminance threshold of a camera sensor for dark tones.

6. The method of claim 5 , wherein the luminance threshold is a luminance value above which a camera sensor response is at least approximately linear.

7. A system comprising:
a processing device; and
a non-transitory computer-readable medium coupled to the processing device, wherein the processing device is configured to execute computer program code stored in the non-transitory computer-readable medium and thereby perform operations comprising:
producing a first reduced-resolution image from a first digital image and a second reduced-resolution image from a second digital image, wherein a first exposure value of the first digital image is greater than a second exposure value of the second digital image, and wherein the first digital image includes a first photographic content item that matches a second photographic content item included in the second digital image;
calculating a light-tone representative scale factor and a dark-tone representative scale factor for each of the first reduced-resolution image and the second reduced-resolution image;
producing a dark-tone empirical scale factor by selectively interpolating between the dark-tone representative scale factor and the light-tone representative scale factor for each of the first reduced-resolution image and the second reduced-resolution image; and
generating an empirically normalized digital image by applying a function including the dark-tone empirical scale factor to each pixel of the first digital image or the second digital image.

8. The system of claim 7 , the operations further comprising calculating the dark-tone representative scale factor by using a dark-tone pairwise set of scale factors from the first reduced-resolution image and the second reduced-resolution image.

9. The system of claim 7 , wherein selectively interpolating comprises interpolating among the light-tone representative scale factor, an updated value of the light-tone representative scale factor, and the dark-tone representative scale factor to update a value of the dark-tone empirical scale factor.

10. The system of claim 9 , the operations further comprising interpolating a normal scale factor based at least in part on exchangeable image file data.

11. The system of claim 7 , wherein the function includes a camera sensor parameter that indicates a luminance threshold of a camera sensor for dark tones.

12. The system of claim 11 , wherein the luminance threshold is a luminance value above which a camera sensor response is at least approximately linear.

13. The system of claim 7 , wherein the first reduced-resolution image from the first digital image and the second reduced-resolution image from the second digital image each have a resolution reduction factor between 5 and 25.

14. A non-transitory computer-readable medium storing program code executable by a processing device to perform operations, the operations comprising:
acquiring a dark exposure digital image and a light exposure digital image, wherein the dark exposure digital image includes a photographic content item that is also included in the light exposure digital image;
producing a first reduced-resolution image from the dark exposure digital image and a second reduced-resolution image from the light exposure digital image;
calculating a light-tone representative scale factor and a dark-tone representative scale factor for each of the first reduced-resolution image and the second reduced-resolution image;
producing a dark-tone empirical scale factor by selectively interpolating between the dark-tone representative scale factor and the light-tone representative scale factor for each of the first reduced-resolution image and the second reduced-resolution image; and
generating an empirically normalized digital image by applying, pixelwise, a function including the dark-tone empirical scale factor to the dark exposure digital image and the light exposure digital image.

15. The non-transitory computer-readable medium of claim 14 , the operations further comprising calculating the dark-tone representative scale factor by using a dark-tone pairwise set of scale factors from the first reduced-resolution image and the second reduced-resolution image.

16. The non-transitory computer-readable medium of claim 14 , wherein the function includes a camera sensor parameter that indicates a luminance threshold of a camera sensor for dark tones.

17. The non-transitory computer-readable medium of claim 16 , wherein the luminance threshold is a luminance value above which a camera sensor response is at least approximately linear.

18. The non-transitory computer-readable medium of claim 14 , wherein selectively interpolating comprises an interpolation of the light-tone representative scale factor, an updated value of the light-tone representative scale factor, and the dark-tone representative scale factor to update a value of the dark-tone empirical scale factor.

19. The non-transitory computer-readable medium of claim 18 , the operations further comprising interpolating a normal scale factor based at least in part on exchangeable image file data.

20. The non-transitory computer-readable medium of claim 14 , wherein the first reduced-resolution image from the dark exposure digital image and the second reduced-re

Assignees

• Adobe Inc

Inventors

• Case Richard

Classifications

• H04N23/951

  by using two or more images to influence resolution, frame rate or aspect ratio · CPC title

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T2207/20208

  High dynamic range [HDR] image processing · CPC title

• G06T2207/20016

  Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform · CPC title