Lightfield rendering based on depths from physically-based volume rendering

Patent metadata

Field	Value
Publication number	US-10893262-B2
Application number	US-201715426616-A
Country	US
Kind code	B2
Filing date	Feb 7, 2017
Priority date	Feb 7, 2017
Publication date	Jan 12, 2021
Grant date	Jan 12, 2021

Abstract

Physically-based volume rendering generates a lightfield. The locations of scattering modeled in physically-based rendering are used to assign depths for the lightfield. The previously assigned depths and previously rendered lightfield are used for lightfield rendering, which may be performed more rapidly than the physically-based volume rendering.

First claim

We claim:

1. A method for lightfield volume rendering, the method comprising:
rendering, by a graphics processing unit or processor as a physically-based renderer and from a medical dataset representing a three-dimensional region of a patient, the medical dataset from a patient scan by a magnetic resonance or computed tomography scanner, to create a lightfield representing the three-dimensional region of the patient in a plurality of two-dimensional views, the rendering stochastically generating a plurality of depths of scattering for each pixel of each of the two-dimensional views;
assigning depths to locations for the pixels of the two-dimensional views in the lightfield, the depth for each pixel of each of the two-dimensional views of the lightfield being determined as a depth combination calculated from the plurality of depths at which stochastically determined scattering used in the rendering by the physically-based renderer for that pixel occurs;
filtering the two-dimensional views as a function of the depths assigned to the pixels of the lightfield;
rendering, by a lightfield renderer performing the filtering, another image representing the three-dimensional region of the patient from the lightfield and depths assigned to the pixels of the lightfield; and
transmitting the other image.

2. The method of claim 1 wherein rendering the lightfield comprises rendering with path tracing.

3. The method of claim 1 wherein rendering the lightfield comprises Monte Carlo-based rendering.

4. The method of claim 1 wherein rendering the lightfield comprises rendering from a plurality of different camera positions relative to the medical dataset.

5. The method of claim 1 wherein assigning comprises, for each pixel of the pixels of the rendered lightfield, locating the depth where the depth combination is an average of the depths of scattering in the physically-based render for the pixel.

6. The method of claim 5 wherein locating comprises locating the depth from a weighted average depth of the scattering, the weighting being a function of color at the depths of scattering being averaged.

7. The method of claim 1 wherein assigning comprises, for each pixel of the pixels of the rendered lightfield, locating the depth where the depth combination is a clustering of sampling points used by the physically-based renderer in rendering the pixel.

8. The method of claim 7 wherein locating the depth with clustering comprises binning depths for the sampling points in a histogram and selecting the depth with a greatest number of hits in the histogram.

9. The method of claim 8 further comprising using multiple depth histograms and sub-dividing the depth histograms.

10. The method of claim 1 wherein assigning comprises generating a plurality of depths for each of at least some of the pixels.

11. The method of claim 1 wherein rendering the other image comprises selecting images of the lightfield with the depths, and interpolating from the selected images.

12. The method of claim 1 wherein filtering comprises using the depths to determine closest rays of the lightfield and wherein rendering comprises generating the other image from the closest rays, where an outlier depth is not included in the depths to determine the closest rays.

13. The method of claim 1 further comprising receiving an interaction from a user input device, the interaction relative to a lightfield image, and wherein rendering the other image comprises rendering from the lightfield based on the interaction.

14. A method for lightfield volume rendering, the method comprising:
rendering, by a graphics processing unit or processor as a physically-based renderer and from a medical dataset representing a three-dimensional region of a patient, to create a lightfield representing the three-dimensional region of the patient in a plurality of two-dimensional views;
assigning depths to locations for pixels of the two-dimensional views in the lightfield, the depths for the pixels of the two-dimensional views of the lightfield each determined from a combination of a plurality of depths at which stochastically determined scattering occurs where the scattering is determined in the rendering by the physically-based renderer;
filtering the two-dimensional views as a function of the depths assigned to the pixels of the lightfield;
rendering, by a lightfield renderer performing the filtering, another image representing the three-dimensional region of the patient from the lightfield and depths; and
transmitting the other image.

Assignees

• Siemens Healthcare Gmbh

Inventors

• Vetter Christoph
• Petkov Kaloian
• Yu Daphne

Classifications

• G06T15/08 Primary

  Volume rendering · CPC title

• G06T2200/21

  involving computational photography · CPC title

• G06T2210/08

  Bandwidth reduction · CPC title

• G06T15/205

  Image-based rendering · CPC title

• G06T2210/41

  Medical · CPC title