Neural networks to render textured materials on curved surfaces

Patent metadata

Field	Value
Publication number	US-2025252655-A1
Application number	US-202519188723-A
Country	US
Kind code	A1
Filing date	Apr 24, 2025
Priority date	Nov 23, 2022
Publication date	Aug 7, 2025
Grant date	—

Abstract

A scene modeling system accesses a three-dimensional (3D) scene including a 3D object. The scene modeling system applies a silhouette bidirectional texture function (SBTF) model to the 3D object to generate an output image of a textured material rendered as a surface of the 3D object. Applying the SBTF model includes determining a bounding geometry for the surface of the 3D object. Applying the SBTF model includes determining, for each pixel of the output image, a pixel value based on the bounding geometry. The scene modeling system displays, via a user interface, the output image based on the determined pixel values.

First claim

1 . A method comprising:
accessing a three-dimensional (3D) scene including one or more 3D objects; generating an output image of a textured material rendered as a surface of a 3D object in the 3D scene through steps comprising:
determining a bounding geometry for the surface of the 3D object;
for each pixel of the output image, determining a pixel value based on the bounding geometry wherein the pixel value comprises a reflectance value, and wherein determining the pixel value for each pixel comprises:
projecting a ray through the pixel into the 3D scene; and
responsive to determining that the ray encounters the bounding geometry, determining the reflectance value for the pixel based in part on a curvature value of the ray and applying the pixel to the surface of the 3D object.

2 . The method of claim 1 , wherein determining the reflectance value comprises:
determining a UV offset for the pixel based on a location on a surface of the bounding geometry; and determining the reflectance value based on the location on the surface of the bounding geometry and the UV offset.

3 . The method of claim 1 , wherein determining the pixel value for each pixel further comprises:
responsive to determining that the ray does not encounter the bounding geometry, assigning an opacity value of zero to the pixel.

4 . The method of claim 3 , wherein generating the output image model further comprises:
responsive to determining that the ray encounters the bounding geometry, determining a non-zero opacity value.

5 . The method of claim 4 , wherein determining the opacity value comprises:
determining a silhouette cosine value; and if a cosine of the ray is less than the silhouette cosine value, determining a zero value for the opacity value; or if the cosine of the ray is greater than or equal to the silhouette cosine value, determining a value of one for the opacity value.

6 . The method of claim 1 , wherein the output image is generated by applying a silhouette bidirectional texture function (SBTF) model to the 3D, the method further comprising training the SBTF model through steps comprising:
generating a training dataset of cylindrical patches of varying radii; applying, to the cylindrical patches of the training dataset, one or more of random rotations, random camera directions, random light directions, or random translations to a UV mapping; and sampling rays incident upon each of the cylindrical patches from different directions perpendicular to a cylinder axis.

7 . A system comprising:
a memory component; and a processing device coupled to the memory component, the processing device to perform operations comprising:
generating an output image of a textured material rendered as a surface of a 3D object in a 3D scene through steps comprising:
determining a bounding geometry for the surface of the 3D object;
for each pixel of the output image, determining a pixel value based on the bounding geometry wherein the pixel value comprises a reflectance value, and wherein determining the pixel value for each pixel comprises:
projecting a ray through the pixel into the 3D scene; and
responsive to determining that the ray encounters the bounding geometry, determining the reflectance value for the pixel based in part on a curvature value of the ray and applying the pixel to the surface of the 3D object.

8 . The system of claim 7 , wherein the output image is generated by applying a silhouette bidirectional texture function (SBTF) model to the 3D object in the 3D scene, wherein the SBTF model comprises:
an alpha model configured to determine an opacity value for a pixel of the output image based at least in part on a bounding geometry of the surface of the 3D object; and a color model configured to determine a reflectance value for the pixel if the opacity value is a non-zero opacity value.

9 . The system of claim 8 , wherein the SBTF model further comprises an offset model configured to determine an offset vector based on a location on the surface of the bounding geometry of a ray projected through the pixel into the 3D scene.

10 . The system of claim 8 , wherein the color model does not determine the reflectance value if the opacity value is zero.

11 . The system of claim 9 , wherein one or more of the alpha model, the offset model, and the color model comprise a fully connected network including a multi-layer perceptron (MLP) that uses a rectified linear unit (ReLU) activation function.

12 . The system of claim 8 , wherein the non-zero opacity value indicates that a ray projected through the pixel into the 3D scene encounters the bounding geometry, and wherein a zero opacity value indicates that the ray does not encounter the bounding geometry.

13 . The system of claim 8 , wherein determining the opacity value comprises:
determining a silhouette cosine value; if a cosine of a ray projected through the pixel into the 3D scene is less than the silhouette cosine value, determining a zero value for the opacity value; or if the cosine of the ray is greater than or equal to the silhouette cosine value, determining a value of one for the opacity value.

14 . The system of claim 8 , wherein the operations further comprise training the SBTF model, wherein training the SBTF model comprises:
generating a training dataset of cylindrical patches of varying radii; applying, to the cylindrical patches of the training dataset, random rotations and translations to a UV mapping; and sampling rays incident upon each of the cylindrical patches from different directions perpendicular to a cylinder axis.

15 . A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising:
accessing a three-dimensional (3D) scene including one or more 3D objects; generating an output image of a textured material rendered as a surface of a 3D object in the 3D scene through steps comprising:
determining a bounding geometry for the surface of the 3D object;
for each pixel of the output image, determining a pixel value based on the bounding geometry wherein the pixel value comprises a reflectance value, and wherein determining the pixel value for each pixel comprises:
projecting a ray through the pixel into the 3D scene; and
responsive to determining that the ray encounters the bounding geometry, determining the reflectance value for the pixel based in part on a curvature value of the ray, and applying the pixel to the surface of the 3D object.

16 . The non-transitory computer-readable medium of claim 15 , wherein generating the output image further comprises determining an opacity value of the pixel, wherein determining the opacity value includes:
determining a silhouette cosine value; determining a zero value for the opacity value if a cosine of the ray is less than the silhouette cosine value; and determining a non-zero value for the opacity value if the cosine of the ray is greater than or equal to the silhouette cosine value.

17 . The non-transitory computer-readable medium of claim 16 , wherein the non-zero opacity value indicates that the ray encounters the bounding geometry, and wherein the zero opacity value indicates that the ray does not encounter the bounding geometry.

18 . The non-transitory computer-readable medium of claim 15 , wherein the pixel value includes a reflectance value, wherein determining the reflectance value comprises:
determining a UV offset for the pixel based on a location on a surface of the b

Assignees

• Adobe Inc
• Univ California

Inventors

• Lakshminarayana Krishna Bhargava Mullia
• Xu Zexiang
• Hasan Milos
• Luan fujun
• Kuznetsov Alexandr
• Wang Xuezheng
• Ramamoorthi Ravi

Classifications

• G06T15/04

  Texture mapping · CPC title

• G06T15/506 Primary

  Illumination models · CPC title

• G06T2215/12

  Shadow map, environment map · CPC title

• G06T15/06 Primary

  Ray-tracing · CPC title