Methods and systems for visualization of 3D parametric data during 2D imaging

Patent metadata

Field	Value
Publication number	US-9330490-B2
Application number	US-201213456967-A
Country	US
Kind code	B2
Filing date	Apr 26, 2012
Priority date	Apr 29, 2011
Publication date	May 3, 2016
Grant date	May 3, 2016

Abstract

Methods and systems for visualization of 3D parametric data in a 2D image. The set of 3D parametric data includes a plurality of voxels in 3D space each associated with at least one parametric value, and the set of 2D image data includes information about a known camera position and a known camera orientation at which the 2D image was obtained. A graphical representation is generated of the parametric values of the voxels corresponding to a viewing surface in 3D space. A virtual 2D view of the viewing surface is determined. The 2D image is displayed registered with the graphical representation of the parametric values of the voxels corresponding to the virtual 2D view.

First claim

The invention claimed is:

1. A method for visualization of 3D parametric data in a 2D image of a non-planar surface, the method comprising:
receiving first signals representing 3D surface data representing a non-planar 3D surface of a physical object to be visualized;
receiving second signals representing a set of 3D parametric data, the 3D parametric data being distinct from the 3D surface data representing the 3D surface of the physical object to be visualized, the set of 3D parametric data including a plurality of voxels in 3D space each associated with at least one parametric value;
spatially registering the 3D surface data representing the 3D surface of the physical object to be visualized and the 3D parametric data, the registered 3D surface having a plurality of surface elements;
receiving third signals representing a set of 2D image data representing the 2D image, the set of 2D image data including information about a known camera position and a known camera orientation at which the 2D image was obtained;
identifying voxels of the set of 3D parametric data corresponding to a set of surface elements of the plurality of surface elements of the 3D surface of the physical object to be visualized, the identifying based at least partly on the spatial registration of the 3D surface data representing the 3D surface of the physical object to be visualized and the 3D parametric data;
generating a graphical representation of the parametric values of the voxels of the set of 3D parametric data corresponding to the set of surface elements of the 3D surface of the physical object to be visualized by, for each surface element in the set of surface elements, assigning a parametric value of the parametric values based on one or more voxels being closest to the respective surface element, and generating a visual representation based on the assigned parametric value;
determining a virtual 2D view of the 3D surface of the physical object to be visualized, the virtual 2D view corresponding to the known camera position and the known camera orientation of the 2D image;
modifying the virtual 2D view with the graphical representation of the parametric values corresponding to the virtual 2D view of the 3D surface of the physical object to be visualized by
combining the obtained 2D image with at least a part of the virtual 2D view to produce a combined 2D image by at least one of overlaying and underlaying the graphical representation of the parametric values with the obtained 2D image, the obtained 2D image and the graphical representation of the parametric values being visible in the combined 2D image based on the visual representations; and
providing signals for displaying the combined 2D image.

2. The method of claim 1 wherein the set of 3D parametric data represents 3D radiation dose data.

3. The method of claim 1 wherein the set of 3D parametric data represents 3D positron emission tomography (PET) data.

4. The method of claim 1 wherein the 3D surface data comprises 3D imaging data, and the spatial registration comprises spatially registering the 3D parametric data to 3D imaging data.

5. The method of claim 4 wherein the 3D imaging data comprises computed tomography (CT) imaging data.

6. The method of claim 1 wherein the obtained 2D image comprises an endoscopic image.

7. The method of claim 1 wherein there is a plurality of 2D images that are updated continuously or intermittently in real-time, the virtual 2D view is updated in real-time to correspond to each update of the 2D images, and the combined 2D image is updated in real-time to display the each updated 2D image registered with the graphical representation corresponding to the updated virtual 2D view.

8. The method of claim 1 further comprising determining which voxels corresponding to the 3D surface are visible at the known camera position and the known camera orientation, where voxels that are visible are determined to be visible voxels and voxels that are not visible are determined to be not visible voxels; and wherein the graphical representation includes only the parametric values of the visible voxels.

9. The method of claim 1 wherein the graphical representation of the parametric values comprises at least one of: a set of isolines and a color map.

10. Previously Presented) The method of claim 9 wherein the graphical representation comprises a set of isolines, and generating the set of isolines comprises: determining a set of isosurfaces in the 3D parametric data, each isosurface including voxels of the 3D parametric data having parametric values equal to a defined value or within a defined value range; and determining, for each isosurface, a respective intersection between that isosurface and the 3D surface, each respective intersection defining an isoline corresponding to the respective defined value or defined value range.

11. The method of claim 9 wherein the graphical representation comprises a color map and generating the color map comprises: generating a copy of the 3D surface having a plurality of defined regions; for each region, determining any intersection between that region and any voxels in the 3D parametric data; where there is no voxel in the 3D parametric data intersecting with that region, assign that region to be transparent; where there is at least one intersecting voxel in the 3D parametric data intersecting with that region, assign an opaque color to that region based on the parametric value of the at least one intersecting voxel; wherein any regions assigned an opaque color together define the color map.

12. The method of claim 1 , wherein a threshold value is defined for the values of the 3D parametric data, wherein any voxel associated with a parametric value that does not meet the threshold value is rendered in a background color or is rendered transparent in the graphical representation.

13. The method of claim 1 , wherein the combining comprises at least one of overlaying and underlaying the obtained 2D image with the at least a part of the updated 2D view.

14. The method of claim 1 wherein the graphical representation of the parametric values is provided for a selected portion of the 3D surface.

15. The method of claim 1 further comprising displaying actual numerical value or values of the parametric value or values for a selected point or portion of the 3D surface.

16. The method of claim 1 wherein the 3D parametric data includes data representative of at least one of a surface property and a subsurface property of the 3D surface.

17. The method of claim 1 , wherein the 3D surface is a tissue surface.

18. The method of claim 1 , wherein generating the graphical representation of the parametric values of the voxels of the set of 3D parametric data corresponding to the set of surface elements of the 3D surface of the physical object to be visualized comprises: generating a set of isolines for 3D continuous contour lines of the 3D surface data with common isovalues, each isoline corresponding to a subset of the surface elements corresponding to the parametric values of the voxels, for each isoline, assigning a unique color to the subset of the surface elements for the respective isoline, and making opaque the set of isolines using the transparency level, the opaque isolines of the graphical representation of the parametric values being visible in the combined 2D image.

19. The method of claim 1 , wherein generating the graphical representation of the parametric values of the voxels of the set of 3D parametric data corresponding to the set of surface ele

Assignees

• Weersink Robert
• Jaffray David A
• Qiu Jimmy
• Hope Andrew
• Cho John
• Sharpe Michael B
• Univ Health Network

Inventors

• Weersink Robert
• Jaffray David A
• Qiu Jimmy
• Hope Andrew
• Cho John
• Sharpe Michael B

Classifications

• A61N5/1002

  Intraluminal radiation therapy · CPC title

• A61B6/542

  involving control of exposure · CPC title

• G06T2219/008

  Cut plane or projection plane definition · CPC title

• A61B6/03

  Computed tomography [CT] · CPC title

• A61B6/5247

  combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound · CPC title