Intensity separated local white balance correction

Patent metadata

Field	Value
Publication number	US-12143733-B2
Application number	US-202318194762-A
Country	US
Kind code	B2
Filing date	Apr 3, 2023
Priority date	Dec 2, 2020
Publication date	Nov 12, 2024
Grant date	Nov 12, 2024

Abstract

Local automatic white balance (AWB) of wide dynamic range (WDR) images is provided. Methods and systems include collecting, by an image signal processor (ISP), statistics for local AWB from at least one wide dynamic range (WDR) image received by the ISP; generating, by a processor, based on the statistics, local gain lookup tables (LUTs), one for each color channel represented in the WDR image(s), each local gain LUT providing a correlation between gain and intensity; and storing the local gain LUTs. Further processing includes, for each of multiple pixels of a WDR image to be output calculating an intensity value, accessing the local gain LUT for the color channel corresponding to that pixel using the calculated intensity value to identify a corresponding local gain value, and applying the local gain value to that pixel.

First claim

What is claimed is:

1. A method, comprising:
collecting, by an image signal processor (ISP), statistics for local automatic white balance (AWB) from at least one wide dynamic range (WDR) image received by the ISP;
generating, by a processor, based on the statistics, a plurality of local gain lookup tables (LUTs), each local gain LUT corresponding to a color channel represented in the at least one WDR image, and each local gain LUT providing a correlation between gain and intensity; and
processing a WDR image, including, for each pixel of multiple pixels of the WDR image:
calculating an intensity value;
determining a local gain LUT of the plurality of local gain LUTs that corresponds to the color channel of the pixel;
identifying a local gain value corresponding to the calculated intensity value of the pixel based on the determined local gain LUT; and
applying the local gain value to the pixel.

2. The method of claim 1 , wherein, for each pixel of the multiple pixels of the WDR image that has a calculated intensity value between two adjacent intensity values in the local gain LUT, the identifying further comprises:
applying linear interpolation between the two adjacent intensity values to identify the corresponding local gain value for the pixel.

3. The method of claim 1 , wherein the calculating further comprises:
calculating the intensity value as a weighted average of an intensity value of the pixel and an intensity value of at least one neighboring pixel.

4. The method of claim 1 , wherein the calculating further comprises:
calculating the intensity value as a weighted average of intensity values of pixels in a 1×6 window of pixels in which the pixel is one of two center pixels in the window.

5. The method of claim 1 , wherein each local gain LUT of the plurality of local gain LUTs includes multiple entries, and wherein each entry comprises an intensity value and a corresponding local gain value.

6. The method of claim 1 , wherein the at least one WDR image from which the statistics are collected includes multiple WDR images, and wherein the collecting statistics further comprises:
collecting first statistics from one WDR image of the multiple WDR images using a first bit range; and
collecting second statistics from another WDR image of the multiple WDR images using a second bit range.

7. A system comprising:
an image signal processor (ISP) configured to receive images and to output processed images, the ISP further configured to collect statistics from at least one image of the images received;
a processor coupled to the ISP, the processor configured to generate, based on the statistics, a plurality of local gain lookup tables (LUTs), one for each color channel represented in the at least one image; and
a memory coupled to the ISP and the processor, the memory configured to store the plurality of local gain LUTs,
wherein the ISP is further configured to process each image to be output including, for each pixel of multiple pixels of the image:
calculating an intensity value;
determining a local gain LUT of the plurality of local gain LUTs corresponding to the color channel associated with the pixel;
determining a local gain value based on the local gain LUT using the calculated intensity value of the pixel; and
applying the local gain value to the pixel.

8. The system of claim 7 , wherein the at least one image from which the statistics are collected includes multiple images, and wherein to collect the statistics, the ISP is configured to collect first statistics from one image of the multiple images using a first bit range and second statistics from another image of the multiple images using a second bit range.

9. The system of claim 7 , wherein to calculate the intensity value, the ISP is configured to calculate the intensity value as a weighted average of an intensity value of the pixel and an intensity value of at least one neighboring pixel.

10. The system of claim 7 , wherein to calculate the intensity value, the ISP is configured to calculate the intensity value as a weighted average of intensity values of pixels in a 1×6 window of pixels in which the pixel is one of two center pixels in the window.

11. The system of claim 7 , wherein each local gain LUT includes multiple intensity values and multiple local gain values respectively corresponding the multiple intensity values.

12. The system of claim 11 , wherein, for each pixel having a calculated intensity value between two adjacent intensity values in the local gain LUT, the ISP is configured to determine the local gain value based on linear interpolation between the two adjacent intensity values.

13. The system of claim 12 , wherein the ISP is configured to perform the linear interpolation using a slope of a segment connecting the two adjacent intensity values.

14. The system of claim 13 , wherein the ISP is further configured to calculate the slope.

15. The system of claim 7 , wherein the system includes a system-on-a-chip (SoC).

16. A device comprising:
a statistics collection component configured to collect statistics for local automatic white balance (LAWB) from multiple wide dynamic range (WDR) images;
memory configured to store a plurality of local gain lookup tables (LUTs), one for each color channel, wherein the plurality of local gain LUTs are generated based on the statistics collected by the statistics collection component, and wherein each of the plurality of local gain LUTs includes multiple local gain values corresponding respectively to multiple intensity ranges; and
a local automatic white balance (LAWB) component configured to: for each pixel of multiple pixels in a WDR image,
calculate an intensity value;
determine a local gain LUT of the plurality of local gain LUTs associated with the color channel corresponding to the pixel;
determine a local gain value based on the local gain LUT using the calculated intensity value; and
apply the local gain value to the pixel.

17. The device of claim 16 , further comprising:
a front-end component configured to receive raw WDR data including multiple exposures and merge at least two exposures into a single WDR image.

18. The device of claim 17 , wherein the front-end component includes the statistics collection component.

19. The device of claim 18 , further comprising:
a brightness and contrast enhancement component coupled to an output of the LAWB component.

20. The device of claim 19 , further comprising:
a color processing component configured to perform color conversion on each WDR image output by the brightness and contrast enhancement component.

Assignees

• Texas Instruments Inc

Inventors

• Hua Gang
• Dabral Shashank
• Mody Mihir Narendra
• Allu Rajasekhar Reddy
• Nandan Niraj

Classifications

• H04N23/741 Primary

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N23/76

  by influencing the image signals · CPC title

• H04N23/71

  Circuitry for evaluating the brightness variation · CPC title

• H04N9/78

  for separating the brightness signal or the chrominance signal from the colour television signal, e.g. using comb filter · CPC title

• H04N23/88 Primary

  for colour balance, e.g. white-balance circuits or colour temperature control · CPC title