Generating non-destructive synthetic lens blur with in-focus edge rendering

Patent metadata

Field	Value
Publication number	US-12561768-B2
Application number	US-202318536812-A
Country	US
Kind code	B2
Filing date	Dec 12, 2023
Priority date	Sep 26, 2023
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

Methods, systems, and non-transitory computer readable storage media are disclosed for generating a lens blur effect in a digital image with in-focus edge rendering. The disclosed system generates a focal matte indicating an in-focus range of depth values of a digital image based on a focus region and a depth map of the digital image. The disclosed system generates a layered depth map comprising foreground depth values and background depth values of pixels across the digital image according to the depth map and the focal matte. The disclosed system also renders the digital image to include a lens blur effect by utilizing the focal matte and the layered depth map to determine a combination of the foreground depth values and the background depth values in connection with a splatting operation.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
generating, by at least one processor, a focal matte indicating an in-focus range of depth values of a digital image based on a focus region and a depth map of the digital image; generating, by the at least one processor, a layered depth map comprising foreground depth values and background depth values of pixels across the digital image according to the depth map and the focal matte; and rendering, by the at least one processor, the digital image to include a lens blur effect by utilizing the focal matte and the layered depth map to determine a combination of the foreground depth values and the background depth values in connection with a splatting operation.

2 . The computer-implemented method of claim 1 , wherein generating the focal matte comprises:
providing, within a graphical user interface, a focal range selection tool comprising a sliding window with an adjustable range of depth values corresponding to a focus relative to a depth map of the digital image; and determining the focus region in response to a position of the sliding window of the focal range selection tool.

3 . The computer-implemented method of claim 2 , wherein generating the focal matte comprises:
providing, within the graphical user interface, the focal range selection tool with one or more options to adjust a size of the sliding window; and determining the focus region based further on the size of the sliding window.

4 . The computer-implemented method of claim 1 , wherein generating the focal matte comprises generating, for the digital image, a focus mask by thresholding depth values in the depth map according to the focus region.

5 . The computer-implemented method of claim 4 , wherein generating the focal matte comprises:
generating, for the digital image and based on the focus region, a trimap segmentation comprising one or more regions corresponding to discontinuities in the depth map; and generating the focal matte by:
utilizing a matting model to generate focal matte values for an unknown region of the trimap segmentation; and
blending the focal matte values of the unknown region with the focus mask.

6 . The computer-implemented method of claim 1 , wherein generating the layered depth map comprises generating, utilizing the focal matte as a guide mask for a refinement machine-learning model, the foreground depth values and the background depth values over all pixels of the digital image according to the in-focus range of depth values.

7 . The computer-implemented method of claim 1 , wherein rendering the digital image to include the lens blur effect comprises:
generating a fused depth map comprising fused depth values of pixels in the digital image by utilizing the focal matte to generate a fused depth value by linearly interpolating between a foreground depth value and a background depth value corresponding to a particular pixel from the layered depth map; and rendering the digital image utilizing the fused depth value of the particular pixel in connection with the splatting operation.

8 . The computer-implemented method of claim 7 , wherein rendering the digital image to include the lens blur effect comprises:
determining splat sizes of the splatting operation for the pixels of the digital image based on fused depth values of the fused depth map; and generating the lens blur effect by integrating splat values for the pixels of the digital image according to the splat sizes.

9 . The computer-implemented method of claim 7 , wherein rendering the digital image to include the lens blur effect comprises:
determining, for the particular pixel, that a focal matte value from the focal matte is greater than a threshold value above zero; and modulating a splat weight for the particular pixel by the focal matte value.

10 . A system comprising:
a memory component; and one or more processing devices coupled to the memory component, the one or more processing devices to perform operations comprising: providing, for display via a graphical user interface, a focal range selection tool comprising an adjustable range of depth values corresponding to a focus relative to a depth map of a digital image; generating a focal matte indicating an in-focus range of depth values of a digital image based on a selected focus region via the focal range selection tool and a trimap segmentation corresponding to discontinuities in the depth map; generating a layered depth map comprising a set of foreground depth values of pixels of the digital image and a set of background depth values of the pixels of the digital image according to the depth map and the focal matte; and render the digital image with a lens blur effect by utilizing the focal matte to determine fused depth values by interpolating between the set of foreground depth values and the set of background depth values of the layered depth map in connection with a splatting operation.

11 . The system of claim 10 , wherein generating the focal matte comprises determining, in response to a user input via the focal range selection tool, a position of a sliding window indicating the adjustable range of depth values corresponding to the focus relative to the depth map.

12 . The system of claim 11 , wherein generating the focal matte comprises modifying a size of the adjustable range of depth values in response to an additional user input changing a position of an edge of the sliding window.

13 . The system of claim 10 , wherein generating the focal matte comprises:
generating a focus mask by thresholding depth values in the depth map according to the selected focus region; generating, based on the focus mask and the depth map, the trimap segmentation indicating an unknown region corresponding to discontinuities in the focus mask; and generating the focal matte by blending focal matte values of the unknown region of the trimap segmentation with the focus mask.

14 . The system of claim 10 , wherein generating the layered depth map comprises generating, utilizing a refinement machine-learning model guided by the focal matte, the set of foreground depth values in a foreground map and the set of background depth values in a background map, the foreground map overlapping the background map.

15 . The system of claim 10 , wherein rendering the digital image to include the lens blur effect comprises:
determining, for a pixel of the digital image, a splat size of the splatting operation based on a fused depth value corresponding to the pixel; and generating the lens blur effect by integrating a splat value for the pixel determined according to the splat size with splat values of adjacent pixels.

16 . The system of claim 10 , wherein rendering the digital image to include the lens blur effect comprises:
determining, for a pixel of the digital image, that a focal matte value from the focal matte is greater than a threshold value; and modulating a splat weight for the pixel by the focal matte value in connection with the splatting operation.

17 . A non-transitory computer readable medium storing executable instructions which, when executed by a processing device, cause the processing device to perform operations comprising:
determining, in response to an input via a graphical user interface, a focus mask comprising an in-focus range of depth values according to a depth map of a digital image indicated by a focus region; generating a focal matte by refining edges of the focus mask according to discontinuities in the depth map; generating a laye

Assignees

• Adobe Inc

Inventors

• Bury Joshua
• Case Richard

Classifications

• G06T7/60

  Analysis of geometric attributes · CPC title

• G06T3/00

  Geometric image transformations in the plane of the image · CPC title

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T2207/20104

  Interactive definition of region of interest [ROI] · CPC title

• G06T5/70 Primary

  Denoising; Smoothing · CPC title