High quality embedded graphics for remote visualization

Patent metadata

Field	Value
Publication number	US-9846926-B2
Application number	US-201514734003-A
Country	US
Kind code	B2
Filing date	Jun 9, 2015
Priority date	Jun 11, 2014
Publication date	Dec 19, 2017
Grant date	Dec 19, 2017

Abstract

Apparatuses and methods for rendering and compositing two-dimensional images for remote visualization using a multi-pass rendering technique are provided. A server computer renders a three-dimensional volume separately from one or more three-dimensional graphical objects to be embedded in the rendered image of the volume. The server compresses and transmits the separately rendered images to a client computer. The client computer decompresses the rendered images and generates a composite image using the rendered images. If a user manipulates, adds or deletes an embedded object, the client computer generates a new composite image using the previously rendered volume image. In the case of a new or manipulated embedded graphical object, the server only renders the manipulated or new object.

First claim

We claim:

1. A method for rendering embedded graphics for remote visualization, the method comprising:
rendering, by a server, a two-dimensional volume rendered image from three-dimensional data captured for a volume;
rendering, by the server, a two-dimensional graphic rendered image to be embedded in the two-dimensional volume rendered image from three-dimensional data for a graphical object, the two-dimensional graphic rendered image only containing volume rendered pixels covered in the two-dimensional volume rendered image by the graphical object;
compressing, by the server, the volume rendered image and the graphic rendered image; and
transmitting, to a workstation over a network, the compressed volume rendered image and the compressed graphic rendered image.

2. The method of claim 1 further comprising:
receiving, by the workstation, the compressed volume rendered image and the compressed graphic rendered image; decompressing, by the workstation, the compressed volume rendered image and the compressed rendered image;
generating, by the workstation, a composite rendered image from the volume rendered image and the graphic rendered image; and
displaying, by the workstation, the composite rendered image.

3. The method of claim 2 further comprising:
receiving, by the workstation, a request to delete the graphic rendered image; and
displaying, by the workstation in response to receiving the request, the volume rendered image without the graphic rendered image.

4. The method of claim 1 wherein the volume rendered image and the graphic rendered image comprise graphics hardware frame buffer objects.

5. The method of claim 4 wherein the graphics hardware frame buffer objects for the graphic rendered image comprise an image set, the image set comprising:
color pixel information of the graphic rendered image;
a depth buffer comprising relative distance information for each pixel of the graphic rendered image; and
a stencil buffer comprising identification of which pixels of the graphic rendered image intersect with pixels of the volume rendered image.

6. The method of claim 5 wherein rendering the graphic rendered image comprises:
rendering, based on the stencil buffer, non-empty pixels for pixels that intersect with pixels of the volume rendered image; and
rendering, based on the stencil buffer, empty pixels for pixels that do not intersect with pixels of the volume rendered image.

7. The method of claim 6 further comprising:
receiving, by the workstation, the volume rendered image and the graphic rendered image;
generating, by the workstation, a composite rendered image from the volume rendered image and the graphic rendered image; and
displaying, by the workstation, the composite rendered image.

8. The method of claim 7 wherein generating the composite rendered image comprises:
replacing pixels of the volume rendered image with non-empty pixels from the graphic rendered image.

9. The method of claim 7 wherein generating the compositing rendered image comprises:
comparing the depth buffer entry for each non-empty pixel from the graphic rendered image with the depth buffer entry for each corresponding pixel for the volume rendered image; and
replacing pixels of the volume rendered image with non-empty pixels from the graphic rendered image with smaller depth buffer values than the corresponding pixels of the volume rendered image.

10. A system for rendering embedded graphics for remote visualization, the system comprising:
a server configured to:
render a two-dimensional volume rendered image from three-dimensional data captured for a volume;
render a two-dimensional graphic rendered image to be embedded in the two-dimensional volume rendered image from three-dimensional data for a graphical object; the two-dimensional graphic rendered image only containing volume rendered pixels covered in the two-dimensional volume rendered image by the graphical object;
compress the volume rendered image and the graphic rendered image;
transmit, over a network, the compressed volume rendered image and the compressed graphic rendered image;

a workstation configured to:
receive, over the network, the compressed volume rendered image and the compressed graphic rendered image;
decompress the compressed volume rendered image and the compressed graphic rendered image;
generate a composite rendered image from the volume rendered image and the graphic rendered image; and
display the composite rendered image.

11. The system of claim 10 wherein the volume rendered image and the graphic rendered image comprise graphics hardware frame buffer objects.

12. The system of claim 11 wherein the graphics hardware frame buffer objects for the graphic rendered image comprise graphic image sets, the graphic image sets comprising:
color pixel information of the graphic rendered image;
a depth buffer comprising relative distance information for each pixel of the graphic rendered image; and
a stencil buffer comprising identification of which pixels of the graphic rendered image intersect with pixels of the volume rendered image.

13. The system of claim 12 wherein the graphic rendered image comprises:
non-empty pixels for pixels that intersect with pixels of the volume rendered image based on the stencil buffer; and
empty pixels for pixels that do not intersect with pixels of the volume rendered image based on the stencil buffer.

14. The system of claim 13 wherein the composite rendered image comprises:
the volume rendered image with pixels replaced by the non-empty pixels from the graphic rendered image.

15. The system of claim 10 wherein the composite rendered image comprises:
the volume rendered image with pixels replaced by non-empty pixels from the graphic rendered image with smaller depth buffer values than the corresponding pixels of the volume rendered image.

16. The system of claim 10 wherein the workstation is further configured to:
receive a request to delete the graphic rendered image; and

display, in response to receiving the request, the volume rendered image without the graphic rendered image.

17. A method for rendering high quality embedded graphics for remote visualization, the method comprising:
scanning a patient to capture three-dimensional volume data representing the patient;
rendering a two-dimensional volume rendered image from the three-dimensional volume data representing the patient;
rendering a plurality of two-dimensional graphic rendered images to be embedded in the two-dimensional volume rendered image from three-dimensional data for a plurality of graphics, wherein the plurality of two-dimensional graphic rendered images only contain volume rendered pixels covered in the two-dimensional volume rendered image by the plurality of graphics; wherein the graphics comprise a display point, a line graphic, a curve, a surface, volume shape, or a combination thereof;
generating a composite rendered image from the volume rendered image and the plurality of graphic rendered images; and
displaying the composite rendered image.

18. The method of claim 17 wherein rendering the plurality graphic rendered images comprises:
rendering non-empty pixels for pixels that intersect with pixels of the volume rendered image; and
rendering empty pixels for pixels that do not intersect with pixels of the volume rendered image.

19. The method of claim 17 wherein generating the composite rendered image comprises:
replacing pixels of the volume render

Assignees

• Siemens Healthcare Gmbh

Inventors

• Hong Wei
• Paulus John
• Qiu Feng
• Yu Daphne

Classifications

• G06T2219/004

  Annotating, labelling · CPC title

• G06T19/00 Primary

  Manipulating three-dimensional [3D] models or images for computer graphics · CPC title

• G06T15/00

  Three-dimensional [3D] image rendering · CPC title

• G06T9/00

  Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) · CPC title

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title