Kernel reshaping-powered splatting-based efficient image space lens blur

Patent metadata

Field	Value
Publication number	US-11869172-B2
Application number	US-202218055161-A
Country	US
Kind code	B2
Filing date	Nov 14, 2022
Priority date	Nov 17, 2020
Publication date	Jan 9, 2024
Grant date	Jan 9, 2024

Abstract

Embodiments are disclosed for generating lens blur effects. The disclosed systems and methods comprise receiving a request to apply a lens blur effect to an image, the request identifying an input image and a first disparity map, generating a plurality of disparity maps and a plurality of distance maps based on the first disparity map, splatting influences of pixels of the input image using a plurality of reshaped kernel gradients, gathering aggregations of the splatted influences, and determining a lens blur for a first pixel of the input image in an output image based on the gathered aggregations of the splatted influences.

First claim

We claim:

1. A computer-implemented method comprising:
receiving, by a user interface manager of a lens blur rendering system, a request to apply a lens blur effect to an input image;
obtaining, by a kernel splatting manager of the lens blur rendering system, a plurality of disparity maps associated with the input image;
aggregating, by the kernel splatting manager, influences of pixels of the input image using a plurality of reshaped kernel gradients, the plurality of reshaped kernel gradients determined based on the plurality of disparity maps; and
determining, by the kernel splatting manager, a lens blur for at least one pixel of the input image in an output image based on the aggregated influences.

2. The computer-implemented method of claim 1 , wherein the request to apply the lens blur effect to the input image includes a first disparity map.

3. The computer-implemented method of claim 2 , wherein the plurality of disparity maps includes a disparity alpha map and a solid disparity map, each based on the first disparity map, and a gradient disparity map, a disparity edge map, and a local min disparity map, each based on the solid disparity map.

4. The computer-implemented method of claim 1 , wherein aggregating, by the kernel splatting manager, influences of pixels of the input image using a plurality of reshaped kernel gradients, the plurality of reshaped kernel gradients determined based on the plurality of disparity maps, further comprises:
splatting the influences using the plurality of reshaped kernel gradients; and
gathering the splatted influences.

5. The computer-implemented method of claim 1 , wherein determining, by the kernel splatting manager of the lens blur rendering system, a lens blur for at least one pixel of the input image in an output image based on the aggregated influences, further comprises:
determining a first color splatting influence excluding occluded content;
determining a second color splatting influence that restrains the influence inside object boundaries;
determining a first count splatting influence excluding occluded content; and
determining a second count splatting influence that restrains the influence inside object boundaries.

6. The computer-implemented method of claim 1 , further comprising:
generating, by the kernel splatting manager, a plurality of distance maps based on the plurality of disparity maps including directional distance maps and a Euclidean distance map.

7. The computer-implemented method of claim 6 , further comprising:
generating, by the kernel splatting manager, the reshaped kernel gradients using the plurality of distance maps.

8. The computer-implemented method of claim 1 , further comprising:
adding highlights to the lens blur based on a highlight map, wherein the highlight map indicates a highlight weight for each pixel of the input image indicating that pixel's relative intensity in the input image.

9. A system comprising:
a computing device implementing a lens blur rendering system, the lens blur rendering system comprising:
a user interface manager receive a request to apply a lens blur effect to an input image; and
a kernel splatting manager to:
obtain a plurality of disparity maps associated with the input image;
aggregate influences of pixels of the input image using a plurality of reshaped kernel gradients, the plurality of reshaped kernel gradients determined based on the plurality of disparity maps; and
determine a lens blur for at least one pixel of the input image in an output image based on the aggregated influences.

10. The system of claim 9 , wherein the request to apply the lens blur effect to the input image includes a first disparity map.

11. The system of claim 10 , wherein the plurality of disparity maps includes a disparity alpha map and a solid disparity map, each based on the first disparity map, and a gradient disparity map, a disparity edge map, and a local min disparity map, each based on the solid disparity map.

12. The system of claim 9 , wherein to aggregate influences of pixels of the input image using a plurality of reshaped kernel gradients, the plurality of reshaped kernel gradients determined based on the plurality of disparity maps, the kernel splatting manager is further to:
splat the influences using the plurality of reshaped kernel gradients; and
gather the splatted influences.

13. The system of claim 9 , wherein to determine a lens blur for at least one pixel of the input image in an output image based on the aggregated influences, the kernel splatting manager is further to:
determine a first color splatting influence excluding occluded content;
determine a second color splatting influence that restrains the influence inside object boundaries;
determine a first count splatting influence excluding occluded content; and
determine a second count splatting influence that restrains the influence inside object boundaries.

14. The system of claim 9 , wherein the kernel splatting manager is further to:
generate a plurality of distance maps based on the plurality of disparity maps including directional distance maps and a Euclidean distance map.

15. The system of claim 14 , wherein the kernel splatting manager is further to:
generate the reshaped kernel gradients using the plurality of distance maps.

16. The system of claim 9 , further comprising:
a highlight manager to add highlights to the lens blur based on a highlight map, wherein the highlight map indicates a highlight weight for each pixel of the input image indicating that pixel's relative intensity in the input image.

17. A system comprising:
means for receiving a request to apply a lens blur effect to an input image;
means for obtaining a plurality of disparity maps associated with the input image;
means for aggregating influences of pixels of the input image using a plurality of reshaped kernel gradients, the plurality of reshaped kernel gradients determined based on the plurality of disparity maps; and
means for determining a lens blur for at least one pixel of the input image in an output image based on the aggregated influences.

18. The system of claim 17 , wherein the request to apply the lens blur effect to the input image includes a first disparity map.

19. The system of claim 18 , wherein the plurality of disparity maps includes a disparity alpha map and a solid disparity map, each based on the first disparity map, and a gradient disparity map, a disparity edge map, and a local min disparity map, each based on the solid disparity map.

20. The system of claim 17 , wherein the means for aggregating influences of pixels of the input image using a plurality of reshaped kernel gradients, the plurality of reshaped kernel gradients determined based on the plurality of disparity maps, further comprises:
means for splatting the influences using the plurality of reshaped kernel gradients; and
means for gathering the splatted influences.

Assignees

• Adobe Inc

Inventors

• Lin Haiting
• Jia Yumin
• Chien Jen-Chan

Classifications

• G06T5/002 Primary

  Physics · mapped topic

• G06T5/20

  using local operators · CPC title

• G06T7/50

  Depth or shape recovery · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06T5/70 Primary

  Denoising; Smoothing · CPC title