Real-time rendering with implicit shapes

Patent metadata

Field	Value
Publication number	US-11875449-B2
Application number	US-202217745478-A
Country	US
Kind code	B2
Filing date	May 16, 2022
Priority date	Nov 30, 2020
Publication date	Jan 16, 2024
Grant date	Jan 16, 2024

Abstract

Systems and methods are described for rendering complex surfaces or geometry. In at least one embodiment, neural signed distance functions (SDFs) can be used that efficiently capture multiple levels of detail (LODs), and that can be used to reconstruct multi-dimensional geometry or surfaces with high image quality. An example architecture can represent complex shapes in a compressed format with high visual fidelity, and can generalize across different geometries from a single learned example. Extremely small multi-layer perceptrons (MLPs) can be used with an octree-based feature representation for the learned neural SDFs.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
dividing a virtual environment, containing a surface to be rendered, into a plurality of voxels, at least one individual voxel of the plurality being associated with one or more feature vectors defining local segments of the surface;
determining a set of query points corresponding to a subset of the plurality of voxels proximate the surface to be rendered;
generating, from the feature vectors associated with the subset of the plurality of voxels, a set of summed feature vectors for the set of query points;
determining, using one or more neural networks and based at least in part upon the set of summed feature vectors, one or more signed distance values for one or more query points of the set of query points; and
rendering the surface using the one or more signed distance values.

2. The computer-implemented method of claim 1 , further comprising:
determining a subset of voxels of the plurality of voxels that contain at least a portion of the surface to be rendered; and
generating a feature volume using the subset of voxels.

3. The computer-implemented method of claim 1 , wherein the plurality of voxels correspond to one or more nodes of a hierarchical tree.

4. The computer-implemented method of claim 3 , wherein the hierarchical tree is a sparse voxel octree (SVO).

5. The computer-implemented method of claim 4 , further comprising:
determining a level of detail (LOD) at which to render the surface, wherein a number of levels in the hierarchical tree is determined based at least in part upon the LOD.

6. The computer-implemented method of claim 1 , wherein the determining the one or more signed distance values comprises using the one or more neural networks to compute one or more neural signed distance functions (SDFs) for the query points.

7. The computer-implemented method of claim 1 , wherein rendering the surface occurs at an interactive display rate.

8. The computer-implemented method of claim 1 , wherein the neural networks are used to represent surfaces other than those used to train the neural networks.

9. The computer-implemented method of claim 1 , further comprising:
performing sphere tracing inside the voxels corresponding to query points in order to determine one or more distance values.

10. A system comprising:
one or more processors to execute operations comprising:
divide a virtual environment containing a surface to be rendered into a plurality of voxels, at least one individual voxel of the plurality being associated with one or more feature vectors defining local segments of the surface;
determine a set of query points corresponding to at least one voxel in a subset of the plurality of voxels and proximate the surface to be rendered,
generate, from the feature vectors associated with the subset of the plurality of voxels, a set of summed feature vectors for the set of query points; and
determine, using one or more neural networks to determine, from the set of summed feature vectors, one or more signed distance values; and
utilizing the signed distance values to render the surface.

11. The system of claim 10 , wherein the one or more processors are further to compute one or more neural signed distance functions (SDFs) for the query points using the one or more neural networks.

12. The system of claim 10 , wherein the voxels corresponding to nodes of a hierarchical tree.

13. The system of claim 12 , wherein the one or more processors are further to determine a level of detail (LOD) at which to render the surface, wherein a number of levels in the hierarchical tree is determined based at least in part upon the LOD.

14. The system of claim 10 , wherein the one or more processors are further to perform sphere tracing inside one or more voxels corresponding to one or more query points in order to determine one or more distance values.

15. The system of claim 10 , wherein the system comprises at least one of:
a system for performing simulation operations;
a system for performing simulation operations to test or validate autonomous machine applications;
a system for rendering graphical output;
a system for performing deep learning operations;
a system implemented using an edge device;
a system incorporating one or more Virtual Machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.

16. A processor comprising:
one or more processing units to divide a virtual environment containing a surface to be rendered, into a plurality of voxels, at least one individual voxel of the plurality being associated with one or more feature vectors defining local segments of the surface, determine a set of query points proximate the surface to be rendered and contained within one or more voxels of a subset of the plurality of voxels, generate, from the feature vectors associated with the subset of the plurality of voxels, a set of summed feature vectors for the set of query points, determine one or more signed distance values using one or more neural networks to determine and based on the set of summed feature vectors, decode the signed distance values to render the surface.

17. The processor of claim 16 , wherein the one or more processing units are further to use neural networks to compute one or more neural signed distance functions (SDFs) for the set of query points.

18. The processor of claim 16 , wherein the plurality of voxels correspond to nodes of a hierarchical tree.

19. The processor of claim 18 , wherein the one or more processing units are further to determine a level of detail (LOD) at which to render the surface, wherein a number of levels in the hierarchical tree is determined based at least in part upon the LOD.

20. The processor of claim 16 , wherein the one or more processing units are further to perform sphere tracing inside voxels containing one or more of the plurality of query points in order to determine the one or more distance values.

Assignees

• Nvidia Corp

Inventors

• Takikawa Towaki Alan
• Litalien Joey
• Yin Kangxue
• Kreis Karsten Julian
• Loop Charles
• Mcguire Morgan
• Fidler Sanja

Classifications

• G06T15/08 Primary

  Volume rendering · CPC title

• G06T17/005

  Tree description, e.g. octree, quadtree · CPC title

• G06T2210/36

  Level of detail · CPC title

• G06T15/10 Primary

  Geometric effects · CPC title

• G06T15/005 Primary

  General purpose rendering architectures · CPC title