Generating shadows for objects in two-dimensional images utilizing a plurality of shadow maps

Patent metadata

Field	Value
Publication number	US-12482172-B2
Application number	US-202318304179-A
Country	US
Kind code	B2
Filing date	Apr 20, 2023
Priority date	Oct 3, 2022
Publication date	Nov 25, 2025
Grant date	Nov 25, 2025

Abstract

The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify two-dimensional images via scene-based editing using three-dimensional representations of the two-dimensional images. For instance, in one or more embodiments, the disclosed systems utilize three-dimensional representations of two-dimensional images to generate and modify shadows in the two-dimensional images according to various shadow maps. Additionally, the disclosed systems utilize three-dimensional representations of two-dimensional images to modify humans in the two-dimensional images. The disclosed systems also utilize three-dimensional representations of two-dimensional images to provide scene scale estimation via scale fields of the two-dimensional images. In some embodiments, the disclosed systems utilizes three-dimensional representations of two-dimensional images to generate and visualize 3D planar surfaces for modifying objects in two-dimensional images. The disclosed systems further use three-dimensional representations of two-dimensional images to customize focal points for the two-dimensional images.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
generating, by at least one processor in response to a request to place a two-dimensional object at a selected position within a scene of a two-dimensional image projected into a three-dimensional space, a shadow map corresponding to the two-dimensional object according to three-dimensional characteristics of the two-dimensional object projected into the three-dimensional space; generating, by the at least one processor, an estimated shadow map for the two-dimensional image based on one or more shadows detected in the two-dimensional image and estimated camera parameters of the two-dimensional image; and generating, by the at least one processor in connection with placing the two-dimensional object at the selected position within the scene of the two-dimensional image, a modified two-dimensional image based on the shadow map corresponding to the two-dimensional object and the estimated shadow map of the two-dimensional image.

2 . The computer-implemented method of claim 1 , wherein generating the shadow map corresponding to the two-dimensional object comprises:
placing a three-dimensional mesh corresponding to the two-dimensional object at a three-dimensional position within the three-dimensional space corresponding to the selected position within the scene of the two-dimensional image; and determining the shadow map corresponding to the two-dimensional object based on the three-dimensional position of the three-dimensional mesh in the three-dimensional space.

3 . The computer-implemented method of claim 2 , wherein placing the three-dimensional mesh corresponding to the two-dimensional object at the three-dimensional position comprises:
determining that the request comprises moving the two-dimensional object from a first position within the scene of the two-dimensional image to a second position within the scene of the two-dimensional image; and generating a proxy three-dimensional mesh representing the two-dimensional object according to features of the two-dimensional object extracted from the two-dimensional image.

4 . The computer-implemented method of claim 3 , wherein generating the shadow map corresponding to the two-dimensional object comprises generating the shadow map based on an estimated camera position of the two-dimensional image, estimated lighting parameters of the two-dimensional image, and the proxy three-dimensional mesh at the three-dimensional position within the three-dimensional space.

5 . The computer-implemented method of claim 2 , wherein placing the three-dimensional mesh corresponding to the two-dimensional object at the three-dimensional position comprises:
determining that the request comprises importing the two-dimensional object into the two-dimensional image for placing at the selected position; and placing an imported three-dimensional mesh representing the two-dimensional object into the two-dimensional image at the three-dimensional position within the three-dimensional space.

6 . The computer-implemented method of claim 5 , wherein generating the shadow map corresponding to the two-dimensional object comprises generating the shadow map based on an estimated camera position of the two-dimensional image and the imported three-dimensional mesh representing the two-dimensional object at the three-dimensional position within the three-dimensional space.

7 . The computer-implemented method of claim 1 , further comprising:
determining an additional two-dimensional object corresponding to the two-dimensional image; generating, in response to the additional two-dimensional object being a different object type than the two-dimensional object, an additional shadow map corresponding to the additional two-dimensional object according to three-dimensional characteristics of the additional two-dimensional object; and generating the modified two-dimensional image based on the shadow map corresponding to the two-dimensional object, the estimated shadow map of the two-dimensional image, and the additional shadow map corresponding to the additional two-dimensional object.

8 . The computer-implemented method of claim 1 , wherein generating the modified two-dimensional image comprises:
determining, based on the estimated camera parameters of the two-dimensional image, a relative positioning of the two-dimensional object and one or more additional two-dimensional objects corresponding to the two-dimensional image according to the three-dimensional characteristics of the two-dimensional object and estimated three-dimensional characteristics of the one or more additional two-dimensional objects; and merging the shadow map of the two-dimensional object and the estimated shadow map of the two-dimensional image based on the relative positioning of the two-dimensional object and the one or more additional two-dimensional objects.

9 . The computer-implemented method of claim 8 , wherein generating the modified two-dimensional image comprises generating, on the two-dimensional object within the modified two-dimensional image, at least a partial shadow from the one or more additional two-dimensional objects in the two-dimensional image or from a scene shadow detected in the scene of the two-dimensional image according to the relative positioning of the two-dimensional object and the one or more additional two-dimensional objects corresponding to the two-dimensional image.

10 . A system comprising:
one or more memory devices comprising a two-dimensional image; and one or more processors configured to cause the system to:
generate, in response to a request to place a two-dimensional object at a selected position within a scene of the two-dimensional image projected into a three-dimensional space, a first shadow map comprising a first shadow type corresponding to the two-dimensional object according to three-dimensional characteristics of the two-dimensional object projected into the three-dimensional space at the selected position;
generate a second shadow map comprising a second shadow type corresponding to the two-dimensional image according to one or more shadows detected in the two-dimensional image and estimated camera parameters of the two-dimensional image; and
generate a modified two-dimensional image comprising the two-dimensional object at the selected position by merging the first shadow map and the second shadow map in connection with the estimated camera parameters of the two-dimensional image.

11 . The system of claim 10 , wherein the one or more processors are configured to cause the system to generate the modified two-dimensional image by:
determining that the two-dimensional object corresponds to an object type comprising a set of object characteristics; and generating the modified two-dimensional image comprising one or more shadows according to the set of object characteristics of the object type.

12 . The system of claim 10 , wherein the one or more processors are configured to cause the system to generate the first shadow map by:
generating a proxy three-dimensional mesh for the two-dimensional object in response to determining that the request comprises moving the two-dimensional object from a first position within the scene of the two-dimensional image to a second position within the scene of the two-dimensional image; and generating the first shadow map based on the proxy three-dimensional mesh and an estimated camera position of the two-dimensional image.

13 . The system of claim 12 , wherein the one or more processors are configured to cause the system to generate the modified two-dimensional image by:
determini

Assignees

• Adobe Inc

Inventors

• Hold-Geoffroy Yannick
• Krs Vojtech
• Mech Radomir
• Carr Nathan
• Gadelha Matheus

Classifications

• G06T2215/12

  Shadow map, environment map · CPC title

• G06T15/60 Primary

  Shadow generation · CPC title