Compressed representation for digital assets

Patent metadata

Field	Value
Publication number	US-2025139878-A1
Application number	US-202318499593-A
Country	US
Kind code	A1
Filing date	Nov 1, 2023
Priority date	Nov 1, 2023
Publication date	May 1, 2025
Grant date	—

Abstract

Techniques for generation of compressed representations for digital assets are described that support computationally efficient and high fidelity rendering of digital assets with a variety of geometries under arbitrary lighting conditions and view directions. A processing device, for instance, receives a digital asset defined by a three-dimensional geometry to be included in a digital scene. The processing device generates a compressed representation of the digital asset that maintains a geometry of the digital asset and includes a precomputed light transport. The processing device then deploys the compressed representation into the digital scene, such as at a location relative to one or more digital scene elements. The content processing system renders the digital asset by applying one or more lighting effects to the compressed representation based on the precomputed light transport and the location relative to the one or more digital scene elements.

First claim

What is claimed is:

1 . A method comprising:
receiving, by a processing device, a digital asset defined by a three-dimensional geometry to be included in a digital scene, the digital scene including at least one digital scene element; generating, by the processing device, a compressed representation of the digital asset, the compressed representation including:
a triplane representation of the three-dimensional geometry; and
a precomputed light transport based on the three-dimensional geometry;
deploying, by the processing device, the compressed representation into the digital scene at a location relative to the at least one digital scene element; and rendering, by the processing device, the digital asset by applying one or more lighting effects to the three-dimensional geometry based on:
the precomputed light transport; and
the location relative to the at least one digital scene element.

2 . The method as described in claim 1 , wherein the triplane representation includes feature grids that correspond to dimensions of the three-dimensional geometry to represent the digital asset.

3 . The method as described in claim 1 , wherein the one or more lighting effects are based in part on a visibility term included in the compressed representation that represents self-occlusion properties of the digital asset.

4 . The method as described in claim 1 , wherein rendering the digital asset includes determining an intersection point of a ray traced from a virtual camera that defines a view direction with the compressed representation and computing a color value to apply to the compressed representation at the intersection point based on the precomputed light transport.

5 . The method as described in claim 4 , wherein the compressed representation further includes a multilayer perceptron, and rendering the digital asset includes:
generating an input vector that corresponds to the intersection point based on the triplane representation; and evaluating the input vector using the multilayer perceptron to determine the color value.

6 . The method as described in claim 5 , wherein the input vector is a concatenation of a feature vector extracted from the triplane representation and a property vector that defines one or more properties of the digital asset at the intersection point.

7 . The method as described in claim 6 , wherein the digital asset is a fiber-based digital asset that includes one or more fiber primitives and the property vector includes data to represent a view direction, a light direction, a tangent, a cross-section offset, and a visibility term particular to the intersection point.

8 . The method as described in claim 6 , wherein the digital asset is a surface-based digital asset and the property vector includes data to represent a view direction, a light direction, a normal, and a visibility term particular to the intersection point.

9 . The method as described in claim 1 , wherein the at least one digital scene element includes one or more of a light source or an additional digital asset.

10 . A system comprising:
a memory component; and a processing device coupled to the memory component, the processing device to perform operations comprising:
receiving a digital asset defined by a three-dimensional geometry to be included in a digital scene, the digital scene including at least one digital scene element;
generating a compressed representation of the digital asset, the compressed representation including:
a triplane representation of the three-dimensional geometry; and
a precomputed light transport based on the three-dimensional geometry;

inserting the compressed representation into the digital scene at a location relative to the at least one digital scene element; and
rendering the digital asset by applying one or more lighting effects to the three-dimensional geometry based on:
the precomputed light transport; and
the location relative to the at least one digital scene element.

11 . The system as described in claim 10 , wherein the triplane representation includes feature grids that correspond to dimensions of the three-dimensional geometry to represent the digital asset.

12 . The system as described in claim 10 , wherein the one or more lighting effects are based in part on a visibility term included in the compressed representation that indicates whether portions of the digital asset are occluded by other portions of the digital asset.

13 . The system as described in claim 12 , the operations further comprising computing the visibility term by tracing an occlusion ray from a first point on the compressed representation to a light source included in the digital scene and determining whether the occlusion ray intersects a second point of the compressed representation.

14 . The system as described in claim 10 , the operations further comprising generating a training dataset that includes training images depicting the digital asset viewed from a plurality of different locations within the digital scene.

15 . The system as described in claim 14 , wherein generating the compressed representation includes utilizing the training dataset to adjust weights of a multi-layer perceptron of the compressed representation and update the triplane representation of the compressed representation.

16 . The system as described in claim 14 , wherein the training dataset includes training samples for each pixel of the training images, the training samples including a visibility term.

17 . The system as described in claim 10 , wherein the at least one digital scene element includes one or more of a light source or an additional digital asset.

18 . A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising:
receiving a digital asset defined by a three-dimensional geometry to be included in a digital scene, the digital scene including at least one additional digital asset; generating a triplane representation of the three-dimensional geometry to be included as part of a compressed representation that includes a precomputed light transport for the digital asset; inserting the compressed representation of the digital asset into a location relative to the at least one additional digital asset within the digital scene; and rendering the digital asset by applying one or more lighting effects to the three-dimensional geometry based on:
the precomputed light transport; and
the location relative to the at least one additional digital asset.

19 . The non-transitory computer-readable medium as described in claim 18 , wherein the one or more lighting effects are based in part on a visibility term included in the compressed representation that represents self-occlusion properties of the digital asset.

20 . The non-transitory computer-readable medium as described in claim 18 , wherein the rendering the digital asset includes using one or more of a path tracer or a rasterizer-based renderer.

Assignees

• Adobe Inc

Inventors

• Mullia Lakshminarayana Krishna Bhargava
• Sun Xin
• Hašan Miloš
• Luan fujun

Classifications

• G06T15/06

  Ray-tracing · CPC title

• G06T15/506 Primary

  Illumination models · CPC title

• G06T15/20

  Perspective computation · CPC title

• G06T15/50 Primary

  Lighting effects · CPC title