Highlight recovery in images

Patent metadata

Field	Value
Publication number	US-10070109-B2
Application number	US-201615198354-A
Country	US
Kind code	B2
Filing date	Jun 30, 2016
Priority date	Jun 30, 2016
Publication date	Sep 4, 2018
Grant date	Sep 4, 2018

Abstract

Embodiments of the present disclosure relate to highlight recovery of a high-resolution image using a single low-resolution image captured at a lower exposure. An example apparatus includes a hue target circuit that receives an input image at a high-resolution including at least one pixel with a clipped color channel. For example, the input image is a Blue sky image with a pixel having clipped Blue channel. The hue target circuit also receives a set of candidate hue maps having a pixel resolution lower than the high-resolution of the input image. The hue target circuit generates a target hue value for the at least one pixel using the pixel information of the set of candidate hue maps. The apparatus also includes a hue recovery circuit that generates a recovered version of the input image by adjusting hue information of the clipped color channel based on the generated target hue.

First claim

What is claimed is:

1. An apparatus, comprising:
a hue target circuit configured to:
receive a first image capturing a scene at a first resolution comprising a pixel with hue information having a clipped color channel;
receive a plurality of candidate hue maps at a second resolution lower than the first resolution, each of the hue maps representing a different color channel of a second image capturing the scene at the second resolution; and
generate a target hue by processing the plurality of candidate hue maps, the target hue including information for recovering the clipped color channel of the pixel; and

a hue recovery circuit configured to generate a recovered version of the first image by adjusting the hue information of the pixel based on the generated target hue.

2. The apparatus of claim 1 , wherein the second image is captured at an exposure level lower than an exposure level of the first image.

3. The apparatus of claim 1 , wherein hue values of the plurality of candidate hue maps corresponding to color channels of pixels of the second image are not clipped.

4. The apparatus of claim 1 , wherein the hue target circuit generates the target hue by:
upsampling the plurality of candidate hue maps into a plurality of upsampled candidate hue maps at the first resolution, and
wherein the target hue is generated based on the upsampled candidate hue maps.

5. The apparatus of claim 4 , wherein the hue target circuit generates the target hue for the pixel by:
determining a weight for each upsampled candidate hue map based on how close the candidate hue value is relative to the clipped channel hue of the pixel, and
wherein the target hue is generated by aggregating weighted candidate hue values.

6. The apparatus of claim 5 , wherein the hue target circuit further generates the target hue for the pixel by:
determining a conservative hue value for the pixel indicated in hue information of the received first image,
wherein the target hue is generated based on the conservative hue value and the aggregated weighted candidate hue values.

7. The apparatus of claim 6 , wherein the hue target circuit generates the target hue value by mixing the conservative hue value with the aggregated weighted candidate hue values.

8. The apparatus of claim 1 , wherein the hue recovery circuit generates the recovered version by:
determining possible recovered values for the pixel using ratio of color channel hue values of the target hues, and
wherein the hue information is adjusted based on the determined possible recovered values for the pixel.

9. The apparatus of claim 8 , wherein the hue recovery circuit generates the recovered version by:
blending the possible recovered values with a hue value of the pixel of the received first image to generate a blended hue value, the blending based on how close the hue value of the pixel is relative to a maximum hue value of an image sensor capturing the first image, and
wherein the hue information is adjusted based on the blended hue value.

10. The apparatus of claim 9 , wherein the hue recovery circuit generates the recovered version by:
performing soft clipping on the blended hue value to generate a soft clipped hue value, and
wherein the hue information is adjusted based on the soft clipped hue value.

11. The apparatus of claim 10 , wherein the hue recovery circuit generates the recovered version by:
performing hue preservation on soft clipped hue value by recovering a hue value altered during the soft clipping to generate a hue preserved pixel value, and
wherein the hue information is adjusted based on the hue preserved pixel value.

12. A method, comprising:
receiving a first image capturing a scene at a first resolution comprising a pixel with hue information having a clipped color channel;
receiving a plurality of candidate hue maps at a second resolution lower than the first resolution, each of the hue maps representing a different color channel of a second image capturing the scene at the second resolution;
generating a target hue by processing the plurality of candidate hue maps, the target hue including information for recovering the clipped color channel of the pixel; and
generating a recovered version of the first image by adjusting the hue information of the pixel based on the generated target hue.

13. The method of claim 12 , wherein the second image is captured at an exposure level lower than an exposure level of the first image.

14. The method of claim 12 , wherein hue values corresponding to color channels of pixels of the second image are not clipped.

15. The method of claim 12 , wherein generating the target hue further comprises:
upsampling the plurality of candidate hue maps into a plurality of upsampled candidate hue maps at the first resolution, and
wherein the target hue is generated based on the upsampled candidate hue maps.

16. The method of claim 15 , wherein generating the target value further comprises:
determining a weight for each upsampled candidate hue map based on how close the candidate hue value is relative to the clipped channel hue of the pixel, and
wherein the target hue is generated by aggregating weighted candidate hue values.

17. The method of claim 12 , wherein generating the recovered version further comprises:
determining possible recovered values for the pixel using a ratio of color channel hue values of the target hues, and
wherein the hue information is adjusted based on the determined possible recovered values for the pixel.

18. A non-transitory computer-readable medium storing a digital representation of an apparatus, the apparatus comprising:
a hue target circuit configured to:
receive a first image capturing a scene at a first resolution comprising a pixel with hue information having a clipped color channel;
receive a plurality of candidate hue maps at a second resolution lower than the first resolution, each of the hue maps representing a different color channel of a second image capturing the scene at the second resolution; and
generate a target hue by processing the plurality of candidate hue maps, the target hue including information for recovering the clipped color channel of the pixel; and

a hue recovery circuit configured to generate a recovered version of the first image by adjusting the hue information of the pixel based on the generated target hue.

19. The apparatus of claim 1 , wherein the first image is a demosaiced version of an image captured by a sensor.

20. The method of claim 12 , wherein the first image is a demosaiced version of an image captured by a sensor.

Assignees

• Apple Inc

Inventors

• Lim Suk Hwan
• Cao Frederic
• Tajbakhsh Touraj

Classifications

• H04N23/84

  for processing colour signals · CPC title

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N9/643 Primary

  Hue control means, e.g. flesh tone control · CPC title

• G06T2207/10024

  Color image · CPC title

• H04N5/2355

  Electricity · mapped topic