Low-poly mesh generation for three-dimensional models

Patent metadata

Field	Value
Publication number	US-12094060-B2
Application number	US-202318517914-A
Country	US
Kind code	B2
Filing date	Nov 22, 2023
Priority date	May 5, 2022
Publication date	Sep 17, 2024
Grant date	Sep 17, 2024

Abstract

In a method, a visual hull is generated based on intersections of first 3D primitives of a plurality of first silhouettes with a bounding box of a 3D model. The first silhouettes are generated by projecting the 3D model onto planes perpendicular to a number of selected view directions of the 3D model. Each of the first 3D primitives is obtained by extruding a connected loop of a respective first silhouette along a view direction of the number of selected view directions that is associated with the respective first silhouette. A carved mesh is be generated based on subtractions of second 3D primitives derived from positive parts of the 3D model to carve out redundant structures from the visual hull. The positive parts are obtained based on fitting planes that slices the 3D model. A low-poly mesh sequence is generated based on progressive simplifications of the carved mesh.

First claim

What is claimed is:

1. A method of generating a low-poly mesh sequence for a three-dimensional (3D) model, the method comprising:
generating a visual hull of an input building 3D model by projecting the input building 3D model onto planes perpendicular to a number of selected view directions of the input building 3D model to generate a plurality of first silhouettes and constructing the visual hull based on intersections of the generated first silhouettes;
forming a carved mesh by removing concave features from the visual hull based on a plurality of fitting planes that slice the input building 3D model; and
generating the low-poly mesh sequence based on progressive simplifications of the carved mesh.

2. The method of claim 1 , wherein the generating the visual hull further comprises:
forming a plurality of fitting planes, a region of the input building 3D model being included in a respective fitting plane of the plurality of fitting planes;
determining a plurality of candidate view directions, each of the plurality of candidate view directions being parallel to a respective pair of fitting planes and associated with a respective weight value, the respective weight value being based on a combined area of regions of the input building 3D model included in the respective pair of fitting planes; and
determining the selected view directions from the plurality candidate view directions that are associated with top k weight values, k being a positive integer.

3. The method of claim 1 , wherein the generating the visual hull further comprises:
simplifying each of the plurality of first silhouettes of the input building 3D model through at least one of a 2D simplification or a shape-size filtering process;
extracting one or more hollow loops from a respective one of the plurality of first silhouettes;
generating one or more connected loops for the respective one of the plurality of first silhouettes by subtracting the one or more hollow loops from a 2D bounding box of the respective one of the plurality of first silhouettes; and
forming first 3D primitives by extruding the one or more connected loops of the plurality of first silhouettes along the number of selected view directions.

4. The method of claim 3 , wherein the generating the visual hull further comprises:
forming a first tentative visual hull by performing a Boolean intersection operation on a first one of the first 3D primitives and a bounding box of the input building 3D model;
determining a first visual difference improvement based on the first tentative visual hull and the input building 3D model;
in response to the visual difference improvement being larger than a threshold value, forming a second tentative visual hull by performing the Boolean intersection operation on a second one of the first 3D primitives and the first tentative visual hull; and
determining a second visual difference improvement based on the second tentative visual hull and the input building 3D model.

5. The method of claim 4 , wherein the determining the first visual difference improvement based on the first tentative visual hull and the input building 3D model further comprises:
determining an initial visual difference based on an averaged pixel-wise difference between the bounding box of the input building 3D model and the input building 3D model;
determining a first visual difference based on an averaged pixel-wise difference between the first tentative visual hull and the input building 3D model; and
determining the first visual difference improvement by subtracting the first visual difference from the initial visual difference.

6. The method of claim 4 , wherein the generating the visual hull further comprises:
determining a n-th tentative visual hull as the visual hull in response to one of (i) a n-th visual difference improvement being smaller than the threshold value and (ii) the n is equal to an upper limit N, N being a positive integer, the n-th tentative visual hull being formed by performing the Boolean intersection operation on a n-th one of the first 3D primitives and a (n−1) th tentative visual hull.

7. The method of claim 1 , wherein the forming the carved mesh further comprises:
slicing the input building 3D model by the plurality of fitting planes, each of the plurality of fitting planes slicing the input building 3D model into a positive part and a negative part, the positive part of the input building 3D model being positioned on a positive side of the respective one of the plurality of fitting planes;
projecting each of the positive parts of the input building 3D model onto a corresponding fitting plane of the plurality of fitting planes to obtain a respective second silhouette;
obtaining an enlarged bounding square of the respective second silhouette on the corresponding fitting plane such that the respective second silhouette is included in the enlarged bounding square;
performing a Boolean subtraction operation to subtract the respective second silhouette from the enlarged bounding square to obtain a boundary loop of the respective second silhouette; and
forming a second 3D primitive by extruding the boundary loop of the respective second silhouette in a normal direction from the positive side of the fitting plane corresponding to the respective second silhouette.

8. The method of claim 1 , wherein the forming the carved mesh further comprises:
forming a first tentative carved mesh by performing a Boolean subtraction operation on a first one of second 3D primitives and the visual hull;
determining a first visual difference improvement based on the first tentative carved mesh and the input building 3D model;
in response to the visual difference improvement being larger than a threshold value, forming a second tentative carved mesh by performing the Boolean subtraction operation on a second one of the second 3D primitives and the first tentative carved mesh; and
determining a second visual difference improvement based on the second tentative carved mesh and the input building 3D model.

9. The method of claim 8 , wherein the forming the carved mesh further comprises:
determining the carved mesh being a n-th tentative carved mesh in response to one of (i) a n-th visual difference improvement being smaller than the threshold value and (ii) the n is equal to a upper limit N, N being a positive integer, the n-th tentative carved mesh being formed by performing a Boolean intersection operation on a n-th one of the second 3D primitives and a (n−1) th tentative carved mesh.

10. The method of claim 1 , wherein the generating the low-poly mesh sequence further comprises:
performing an edge collapse and edge flip operation on the carved mesh progressively to generate the low-poly mesh sequence, each of the low-poly meshes in the low-poly mesh sequence including a respective number of triangles less than a user selected value T, the T being a positive integer, a number of triangles of each of the low-poly meshes is less than a number of triangles in the carved mesh.

11. The method of claim 1 , further comprising:
ranking the low-poly meshes in the low-poly mesh sequence based on a number of faces and visual differences of the low-poly meshes to generate a Pareto set.

12. An apparatus for generating a low-poly mesh sequence for a three-dimensional (3D) model, comprising:
processing circuitry configured to:
generate a visual hull of an input building 3D model by projecting the input building 3D model onto planes perpendicular to a number of selected view directions of the input building 3D model to generate a plurality of first silhouettes and constructing the visual hull based on intersections of t

Assignees

• Tencent America LLC

Inventors

• Gao Xifeng
• Wu Kui
• Pan Zherong

Classifications

• G06T2210/12

  Bounding box · CPC title

• G06T17/10

  Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes · CPC title

• G06T15/205

  Image-based rendering · CPC title

• G06T17/205 Primary

  Re-meshing · CPC title