Generating soft object shadows for general shadow receivers within digital images using geometry-aware buffer channels

Patent metadata

Field	Value
Publication number	US-2024273813-A1
Application number	US-202318168995-A
Country	US
Kind code	A1
Filing date	Feb 14, 2023
Priority date	Feb 14, 2023
Publication date	Aug 15, 2024
Grant date	—

Abstract

The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object shadows for digital images utilizing corresponding geometry-aware buffer channels. For instance, in one or more embodiments, the disclosed systems generate, utilizing a height prediction neural network, an object height map for a digital object portrayed in a digital image and a background height map for a background portrayed in the digital image. The disclosed systems also generate, from the digital image, a plurality of geometry-aware buffer channels using the object height map and the background height map. Further, the disclosed systems modify the digital image to include a soft object shadow for the digital object using the plurality of geometry-aware buffer channels.

First claim

What is claimed is:

1 . A method comprising:
generating, utilizing a height prediction neural network, an object height map for a digital object portrayed in a digital image and a background height map for a background portrayed in the digital image; generating, from the digital image, a plurality of geometry-aware buffer channels using the object height map and the background height map; and modifying the digital image to include a soft object shadow for the digital object using the plurality of geometry-aware buffer channels.

2 . The method of claim 1 , wherein generating, from the digital image, the plurality of geometry-aware buffer channels using the object height map and the background height map comprises generating a first geometry-aware buffer channel that includes a first gradient of the background height map in a first direction.

3 . The method of claim 2 , wherein generating, from the digital image, the plurality of geometry-aware buffer channels using the object height map and the background height map comprises generating a second geometry-aware buffer channel that includes a second gradient of the background height map in a second direction.

4 . The method of claim 1 , wherein generating, from the digital image, the plurality of geometry-aware buffer channels using the object height map and the background height map comprises generating a geometry-aware buffer channel that includes relative distances for pixels of the background and corresponding pixels of the digital object utilizing the object height map.

5 . The method of claim 1 , wherein modifying the digital image to include the soft object shadow for the digital object using the plurality of geometry-aware buffer channels comprises modifying the digital image utilizing a shadow stylization neural network to include the soft object shadow for the digital object based on the plurality of geometry-aware buffer channels.

6 . The method of claim 1 ,
further comprising determining a softness value for the soft object shadow, wherein modifying the digital image to include the soft object shadow for the digital object using the plurality of geometry-aware buffer channels comprises modifying the digital image to include the soft object shadow for the digital object using the plurality of geometry-aware buffer channels and the softness value.

7 . The method of claim 1 , wherein generating, from the digital image, the plurality of geometry-aware buffer channels comprises generating, from the digital image, one or more geometry-aware buffer channels using at least one of a position of a light source associated with the digital image or a horizon associated with the digital image.

8 . The method of claim 1 , wherein generating, from the digital image, the plurality of geometry-aware buffer channels using the object height map and the background height map comprises generating, from the digital image, a geometry-aware buffer channel that includes sparse hard object shadows cast by the digital object in accordance with an area light source associated with digital image.

9 . A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising:
determining, utilizing a height prediction neural network, an object height map for a digital object portrayed in a digital image and a background height map for a background portrayed in the digital image; determining, from the digital image, a plurality of geometry-aware buffer channels using the object height map and the background height map; and modifying the digital image to include a soft object shadow for the digital object using the plurality of geometry-aware buffer channels.

10 . The non-transitory computer-readable medium of claim 9 , wherein determining, from the digital image, the plurality of geometry-aware buffer channels using the object height map and the background height map comprises generating a geometry-aware buffer channel that includes relative pixel height differences for pixels of the background and corresponding pixels of the digital object utilizing the object height map and the background height map.

11 . The non-transitory computer-readable medium of claim 10 , wherein determining the geometry-aware buffer channel that includes the relative pixel height differences for the pixels of the background and the corresponding pixels of the digital object utilizing the object height map and the background height map comprises:
determining a first pixel height for a pixel of the background using the background height map; determining a second pixel height for a pixel of the digital object that corresponds to the pixel of the background using the object height map; and determining a height difference between the first pixel height and the second pixel height.

12 . The non-transitory computer-readable medium of claim 11 , wherein the operations further comprise determining the pixel of the digital object that corresponds to the pixel of the background by determining that the pixel of the digital object is blocking light from reaching the pixel of the background.

13 . The non-transitory computer-readable medium of claim 9 , wherein:
determining, from the digital image, a plurality of geometry-aware buffer channels using the object height map and the background height map comprises generating a geometry-aware buffer channel that includes a hard object shadow for the digital object using the object height map and the background height map; and modifying the digital image to include the soft object shadow for the digital object using the plurality of geometry-aware buffer channels comprises modifying the hard object shadow of the geometry-aware buffer channel to generate the soft object shadow within the digital image.

14 . The non-transitory computer-readable medium of claim 9 , wherein determining, from the digital image, the plurality of geometry-aware buffer channels using the object height map and the background height map comprises generating geometry-aware buffer channels that are translation invariant by generating one or more geometry-aware buffer channels that include a gradient of the background height map.

15 . The non-transitory computer-readable medium of claim 9 , wherein determining, from the digital image, the plurality of geometry-aware buffer channels using the object height map and the background height map comprises generating a geometry-aware buffer channel that includes relative distances for pixels of the background and corresponding pixels of the digital object utilizing the object height map, a relative distance for a pixel of the background and a corresponding pixel of the digital object comprising a distance between the pixel of the background and a ground point corresponding to the pixel of the digital object.

16 . A system comprising:
one or more memory components; and one or more processing devices coupled to the one or more memory components, the one or more processing devices to perform operations comprising:
receiving a digital image portraying a digital object against a non-planar background;
generating, utilizing a height prediction neural network, an object height map for the digital object and a background height map for the non-planar background of the digital image;
generating a first geometry-aware buffer channel that includes a hard object shadow for the digital object using the object height map;
generating a second geometry-aware buffer channel that includes a gradient of the background height

Assignees

• Adobe Inc

Inventors

• Zhang Jianming
• Sheng Yichen
• Philip Julien
• Hold-Geoffroy Yannick
• Sun Xin
• Zhang He

Classifications

• G06T15/60 Primary

  Shadow generation · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06V10/60 Primary

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06T7/60

  Analysis of geometric attributes · CPC title