Semantic cinematic volume rendering

Patent metadata

Field	Value
Publication number	US-9799135-B2
Application number	US-201514842280-A
Country	US
Kind code	B2
Filing date	Sep 1, 2015
Priority date	Sep 1, 2015
Publication date	Oct 24, 2017
Grant date	Oct 24, 2017

Abstract

The present embodiments relate to cinematic volume renderings and volumetric Monte-Carlo path tracing. The present embodiments include systems and methods for integrating semantic information into cinematic volume renderings. Scan data of a volume is captured by a scanner and transmitted to a server or workstation for rendering. The scan data is received by a server or workstation. The server or workstation extracts semantic information and/or applies semantic processing to the scan data. A cinematic volume rendering is generated from the scan data and the extracted semantic information.

First claim

We claim:

1. A method for generating a volume rendering, the method comprising:
receiving, by a server over a network, a three-dimensional data set of a volume;
extracting, by the server, semantic information from the three-dimensional data set;
generating, by the server, a volume rendering from the three-dimensional data set and from the extracted semantic information, wherein generating the volume rendering comprises performing volumetric Monte-Carlo path tracing using the three-dimensional data set and integrating the extracted semantic information; and
displaying the generated volume rendering.

2. The method of claim 1 wherein extracting semantic information comprises extracting material properties of the volume from the three-dimensional data set.

3. The method of claim 2 wherein extracting material properties comprises associating the three-dimensional data with tissue classifications.

4. The method of claim 1 wherein extracting semantic information comprises extracting surface characteristics of the volume from the three-dimensional data set.

5. The method of claim 4 wherein extracting surface characteristics comprises segmenting a structure of the volume from the three-dimensional data set.

6. The method of claim 1 wherein extracting semantic information comprises extracting semantic information from the three-dimensional data set to apply an illumination model.

7. The method of claim 6 wherein applying the illumination model comprises generating a feature map for the volume from the three-dimensional data set.

8. The method of claim 7 wherein generating a feature map comprises generating a hot spot map.

9. The method of claim 1 wherein extracting semantic information comprises two or more levels extraction performed on the three-dimensional data set, wherein the levels of extraction comprise two or more of:
extracting material properties of the volume from the three-dimensional data set;
extracting surface characteristics of the volume from the three-dimensional data set; and
extracting semantic processing from the three-dimensional data set to apply an illumination model.

10. A method for generating a volume rendering, the method comprising:
receiving a plurality of voxels representing a volume, wherein each voxel has three-dimensional coordinates and an image intensity associated with the voxel;
applying semantic processing to one or more voxels to extract semantic information from the voxels;
generating a volume rendering based on the plurality of voxels and the extracted semantic information, wherein generating the volume rendering comprises performing volumetric Monte-Carlo path tracing using the three-dimensional data set and the extracted semantic information; and
displaying the generated volume rendering.

11. The method of claim 10 wherein applying semantic comprises associating a material property, associating a surface characteristic or applying illumination model to the voxels.

12. The method of claim 10 wherein associating one or more voxels with semantic information comprises labeling the voxels with one or more semantic labels.

13. The method of claim 12 wherein labeling the voxel further comprises labeling each voxel with a semantic probability associated with each semantic label.

14. The method of claim 10 , wherein applying semantic processing comprises filtering the voxels based the image intensity associated with the voxel, wherein filtering the voxels comprises labeling each voxel with a semantic label.

15. The method of claim 14 , wherein labeling each voxel with a semantic label comprises labeling each voxel with a material property, wherein the image intensity and the material property label are used to generate a transfer function for the voxels.

16. The method of claim 14 , wherein labeling each voxel with a semantic label comprises labeling each voxel with a surface property, wherein the image intensity and the surface property label are used to generate a phase function for the voxels.

17. The method of claim 14 , wherein labeling each voxel with a semantic label comprises labeling each voxel with a material property, wherein the material property label is used to apply a light model to the voxels.

18. The method of claim 17 , wherein applying a light model to the voxels generates a feature map.

19. A system for generating a volume rendering, the system comprising:
a scanner configured to capture scan data of a volume; and
a server configured to:
receive, from the scanner over a network, the scan data of a volume;
extract semantic information from the scan data, wherein the semantic information comprises a material property, a surface characteristic or an illumination property; and
generate, using volumetric Monte-Carlo path tracing, a volume rendering incorporating the semantic information.

20. The system of claim 19 further comprising:
a workstation configured to:
receive, from the server over the network, the generated volume rendering; and
display the generated volume rendering.

Assignees

• Siemens Healthcare Gmbh

Inventors

• Zhou Shaohua Kevin
• Engel Klaus
• Liu David
• Yu Daphne
• Geiger Bernhard
• Lay Nathan

Classifications

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2415

  based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate · CPC title

• G06V10/56

  relating to colour · CPC title

• G06K9/4652

  Physics · mapped topic

• G06T15/08 Primary

  Volume rendering · CPC title