Methods for a rasterization-based differentiable renderer for translucent objects

Patent metadata

Field	Value
Publication number	US-12148095-B2
Application number	US-202217932640-A
Country	US
Kind code	B2
Filing date	Sep 15, 2022
Priority date	Sep 15, 2022
Publication date	Nov 19, 2024
Grant date	Nov 19, 2024

Abstract

Systems and methods for rendering a translucent object are provided. In one aspect, the system includes a processor coupled to a storage medium that stores instructions, which, upon execution by the processor, cause the processor to receive at least one mesh representing at least one translucent object. For each pixel to be rendered, the processor performs a rasterization-based differentiable rendering of the pixel to be rendered using the at least one mesh and determines a plurality of values for the pixel to be rendered based on the rasterization-based differentiable rendering. The rasterization-based differentiable rendering can include performing a probabilistic rasterization process along with aggregation techniques to compute the plurality of values for the pixel to be rendered. The plurality of values includes a set of color channel values and an opacity channel value. Once values are determined for all pixels, an image can be rendered.

First claim

The invention claimed is:

1. A method for use in rendering a translucent object, the method comprising:
receiving at least one mesh representing at least one translucent object;
determining a pixel to be rendered;
determining a set of triangles from the at least one mesh based on the pixel to be rendered;
performing probabilistic rasterization for the pixel to be rendered by computing a plurality of probability values, wherein each probability value is computed based on a location of the pixel to be rendered and an edge of a triangle within the determined set of triangles;
grouping the determined set of triangles based on a depth difference threshold into subsets of grouped triangles;
computing a plurality of fragments using the computed set of probability values, wherein each fragment includes a set of fragment color channel values and a fragment opacity channel value computed by aggregating a subset of grouped triangles; and
computing a set of color channel values and an opacity channel value for the pixel to be rendered by aggregating the plurality of fragments.

2. A computer system for rendering a translucent object, the computer system comprising:
a processor coupled to a storage medium that stores instructions, which, upon execution by the processor, cause the processor to:
receive at least one mesh representing at least one translucent object;
for each pixel to be rendered:
perform a rasterization-based differentiable rendering of the pixel to be rendered using the at least one mesh by:
performing a probabilistic rasterization by computing a set of probability values; and

determine a plurality of values for the pixel to be rendered based on the set of probability values, wherein the plurality of values includes a set of color channel values and an opacity channel value; and

output a rendered image based on the determined pluralities of values.

3. The system of claim 2 , wherein performing the probabilistic rasterization comprises:
determining a set of polygons from the at least one mesh based on the pixel to be rendered; and
computing the set of probability values associated with the determined set of polygons, wherein each of the probability values is computed based on a location of the pixel to be rendered and an edge of a polygon within the determined set of polygons.

4. The system of claim 3 , wherein each of the probability values is computed using a sigmoid function.

5. The system of claim 3 , wherein performing the rasterization-based differentiable rendering further includes grouping the determined set of polygons based on a depth difference threshold into subsets of grouped polygons.

6. The system of claim 5 , wherein performing the rasterization-based differentiable rendering further includes computing a plurality of fragments, wherein each fragment includes a set of fragment color channel values and a fragment opacity channel value computed by aggregating a subset of grouped polygons.

7. The system of claim 6 , wherein:
each polygon in the determined set of polygons includes a set of polygon color channel values and a polygon opacity channel value;
each fragment color channel value is computed based on a weighted average of a corresponding polygon color channel value from each of the respective grouped polygons; and
each fragment opacity channel value is computed based on a weighted average of a corresponding polygon opacity value from each of the respective grouped polygons.

8. The system of claim 7 , wherein:
a weight of P is used to compute the weighted average of the corresponding polygon color channel values; and
a weight of min(2P, 1) is used to compute the weighted average of the corresponding polygon opacity value, where P is the probability value associated with the respective polygon.

9. The system of claim 6 , wherein the plurality of values for the pixel to be rendered is computed by aggregating the plurality of fragments.

10. The system of claim 9 , wherein:
the opacity channel value in the plurality of values for the pixel to be rendered is computed as:

alpha
=

1
-

∏

i
=
1

n

(

1
-

alpha
i

)

,

wherein alpha is the computed opacity channel value, n is the number of fragments in the plurality of fragments, and alpha i is a fragment opacity channel value; and
the set of color channel values in the plurality of values for the pixel to be rendered is computed as:

color
=

1
alpha

⁢

∑

i
=
1

n

[

alpha
i

×

color
i

×

∏

j
=
1

i
=
1

(

1
-

alpha
j

)

]

,

wherein color is the color channel value that is to be computed, color i is a fragment color channel value, and alpha j is a fragment opacity channel value.

11. The system of claim 2 , wherein the set of color channel values includes a red color channel value, a green color channel value, and a blue color channel value.

12. A method for rendering a translucent object, the method comprising:
receiving at least one mesh representing at least one translucent object;
for each pixel to be rendered:
performing a rasterization-based differen

Assignees

• Lemon Inc

Inventors

• Zhi Tiancheng
• Sang Shen
• Song Guoxian
• Lai Chunpong
• Liu Jing
• Luo Linjie

Classifications

• G06T17/10

  Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes · CPC title

• G06T2210/62

  Semi-transparency · CPC title

• G06T17/20 Primary

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G06T15/506

  Illumination models · CPC title

• G06T15/20

  Perspective computation · CPC title