Regularized derivative operators for image processing system and method

Patent metadata

Field	Value
Publication number	US-11238573-B2
Application number	US-202016900653-A
Country	US
Kind code	B2
Filing date	Jun 12, 2020
Priority date	Jun 13, 2019
Publication date	Feb 1, 2022
Grant date	Feb 1, 2022

Abstract

Devices, methods, and non-transitory program storage devices are disclosed herein to provide improved image processing, the techniques comprising: obtaining an input image and target image data, and then calculating derivatives for the target image data using a regularized derivative kernel operator. In some embodiments, the regularized operator may comprise the following operator: [−1 (1+ε)], wherein ε may be a controllable system parameter and preferably is independent of the particular type of image processing being applied to the image. In some embodiments, the techniques may find look-up table (LUT) mappings or analytical functions to approximate the derivative structure of the target image data. Finally, the techniques disclosed herein may generate an output image from the input image based on attempting to closely approximate the calculated derivatives for the target image data. In preferred embodiments, by controlling the mapping, e.g., using regularization techniques, halos and other image artifacts may be ameliorated.

First claim

What is claimed is:

1. A device, comprising:
a memory;
one or more image capture devices;
a user interface; and
one or more processors operatively coupled to the memory, wherein the one or more processors are configured to execute instructions causing the one or more processors to:
obtain an input image and target image data;
calculate derivatives for the target image data using a regularized derivative operator, wherein the regularized derivative operator is fixed over the target image data and configured to perform regularization at an individual pixel level; and
generate an enhanced output image from the input image, wherein the enhanced output image is determined based on the calculated derivatives for the target image data.

2. The device of claim 1 , wherein the target image data comprises gradient values.

3. The device of claim 1 , wherein the instructions further comprise instructions to:
determine a Look-up table (LUT) comprising mappings from the input image determined based on a derivative structure of the target image data, and
wherein the enhanced output image is further determined by using the determined LUT.

4. The device of claim 3 , wherein the instructions to determine a Look-up table (LUT) comprising mappings from the input image determined based on a derivative structure of the target image data further comprise instructions to:
determine a Look-up table (LUT) comprising mappings from the input image determined based on solving a minimization operation attempting to match a derivative structure of the input image to the derivative structure of the target image data.

5. The device of claim 3 , wherein the instructions to determine a Look-up table (LUT) comprising mappings from the input image determined based on a derivative structure of the target image data further comprise instructions to:
determine a Look-up table (LUT) using downscaled versions of the input image and the target image data.

6. The device of claim 1 , wherein the instructions to generate an enhanced output image from the input image data, wherein the enhanced output image is determined based on the calculated derivatives for the target image data further comprise instructions to:
generate an enhanced output image from the input image data, wherein the enhanced output image is determined based on solving a minimization operation attempting to match a derivative structure of the input image to the derivative structure of the target image data.

7. The device of claim 1 , wherein the instructions to generate an enhanced output image from the input image data, wherein the enhanced output image is determined based on the calculated derivatives for the target image data further comprise instructions to:
map the input image to the enhanced output image using one or more analytic functions.

8. The device of claim 1 , wherein the regularized derivative operator comprises a kernel operator.

9. The device of claim 8 , wherein the kernel operator is defined as: [−a, b], wherein b>a (wherein a>0 and b>0).

10. The device of claim 8 , wherein the kernel operator is defined as: [−1(1+∈)], wherein ∈>0.

11. The device of claim 10 , wherein the ∈ parameter is a controllable system parameter that is independent of a type of image processing being applied to the input image.

12. A non-transitory computer readable medium comprising computer readable instructions configured to cause one or more processors to:
obtain an input image and target image data;
calculate derivatives for the target image data using a regularized derivative operator, wherein the regularized derivative operator is fixed over the target image data and configured to perform regularization at an individual pixel level; and
generate an enhanced output image from the input image, wherein the enhanced output image is determined to be an image having derivatives based on the calculated derivatives for the target image data.

13. The non-transitory computer readable medium of claim 12 , wherein the instructions further comprise instructions to:
determine a Look-up table (LUT) comprising mappings from the input image determined based on a derivative structure of the target image data, and
wherein the enhanced output image is further determined by using the determined LUT.

14. The non-transitory computer readable medium of claim 12 , wherein the regularized derivative operator comprises a kernel operator defined as: [−1 (1+∈)], wherein ∈>0.

15. The non-transitory computer readable medium of claim 14 , wherein the ∈ parameter is a controllable system parameter that is independent of a type of image processing being applied to the input image.

16. An image processing method, comprising:
obtaining an input image and target image data;
calculating derivatives for the target image data using a regularized derivative operator, wherein the regularized derivative operator is fixed over the target image data and configured to perform regularization at an individual pixel level; and
generating an enhanced output image from the input image, wherein the enhanced output image is determined to be an image having derivatives based on the calculated derivatives for the target image data.

17. The method of claim 16 , further comprising:
determining a Look-up table (LUT) comprising mappings from the input image determined based on a derivative structure of the target image data, and
wherein the enhanced output image is further determined by using the determined LUT.

18. The method of claim 16 , wherein generating the enhanced output image from the input image further comprises using a Poisson-type solver operation.

19. The method of claim 18 , wherein the regularized derivative operator comprises a kernel operator defined as: [−1 (1+∈)], wherein ∈>0.

20. The method of claim 16 , wherein calculating derivatives for the target image data using a regularized derivative operator further comprises:
calculating x-derivatives of the target image data by differentiation using a derivative kernel; and
calculating y-derivatives of the target image data by differentiation using a derivative kernel,
wherein generating an enhanced output image from the input image further comprises generating an enhanced output image determined to have derivatives based on the calculated x-derivatives and y-derivatives of the target image data.

Assignees

• Apple Inc

Inventors

• Finlayson Graham
• Drew Mark

Classifications

• G06T5/92

  based on global image properties · CPC title

• G06T5/94 Primary

  based on local image properties, e.g. for local contrast enhancement · CPC title

• G06T2207/20208

  High dynamic range [HDR] image processing · CPC title

• G06T5/20

  using local operators · CPC title

• G06T5/001

  Physics · mapped topic