System and method for providing a contour video with a 3D surface in a medical navigation system

Patent metadata

Field	Value
Publication number	US-10543045-B2
Application number	US-201515575552-A
Country	US
Kind code	B2
Filing date	Jul 13, 2015
Priority date	Jul 13, 2015
Publication date	Jan 28, 2020
Grant date	Jan 28, 2020

Abstract

A medical navigation system for displaying a three dimensional (3D) surface video of a target is provided. The medical navigation system comprises a 3D imaging device, a camera, a display, and a controller electrically coupled to the 3D imaging device, the camera, and the display. The controller has a processor coupled to a memory. The controller is configured to perform calibration of input devices; acquire 3D depth data of the target from a signal generated by the 3D imaging device; construct a 3D surface contour of the target based on the 3D depth data; acquire a video stream of the target from a signal generated by the camera; generate a 3D surface video based on the 3D surface contour and the video stream; and display the 3D surface video on the display.

First claim

We claim:

1. A medical navigation system for displaying a three dimensional (3D) surface video of a target, the medical navigation system comprising:
a structured light 3D scanner;
a camera;
a display; and
a controller electrically coupled to the 3D scanner, the camera, and the display, the controller having a processor coupled to a memory, the controller being configured to:
acquire the 3D depth data of the target from a signal generated by the 3D scanner;
construct a 3D surface contour, including a 3D point cloud, of the target based on the 3D depth data;
acquire a video stream of the target from a signal generated by the camera;
generate a 3D surface video based on the 3D surface contour and the video stream; and
display the 3D surface video on the display;
wherein the 3D surface contour is used by the controller to exclude occlusion events such that an object passing between the target and at least one of the 3D scanner and the camera is not visible in the 3D surface video.

2. The medical navigation system according to claim 1 , further comprising:
a positioning device having a positioning arm with an end effector at the end of the positioning arm, the positioning device electrically coupled to the controller and at least one of the 3D scanner and the camera being mountable on the end effector.

3. The medical navigation system according to claim 1 , wherein the camera is selected from a group consisting of a video camera, an infrared camera, a visible light camera, and a non-visible light camera.

4. The medical navigation system according to claim 1 , wherein the 3D surface video displayed on the display shows a 3D video that is dynamically rotatable about any axis.

5. The medical navigation system according to claim 1 , wherein the display includes a two dimensional video display.

6. The medical navigation system according to claim 1 , wherein the display includes a stereo display system.

7. The medical navigation system according to claim 1 , wherein generating the 3D surface video includes coloring each point of the 3D point cloud using colour provided by the video stream.

8. The medical navigation system according to claim 1 , wherein any objects having a depth that is beyond a threshold depth distance outside of the 3D surface contour are not shown in the 3D surface video.

9. The medical navigation system according to claim 1 , wherein the target includes human tissue.

10. The medical navigation system according to claim 1 , further comprising input devices including at least one of the 3D scanner, the camera, and a tracking system of the medical navigation system, the controller being further configured to:
perform calibration of input devices by mapping coordinates of the input devices into a common coordinate system.

11. The medical navigation system according to claim 1 , wherein displaying the 3D surface video on the display includes overlaying the 3D surface video onto a corresponding portion of pre-operative images displayed on the display.

12. A method for displaying a three dimensional (3D) surface video of a target in a system having a structured light 3D scanner, a camera, a display, and a controller electrically coupled to the 3D scanner, the camera, and the display, the method comprising:
acquiring the 3D depth data of the target from a signal generated by the 3D scanner;
constructing a 3D surface contour, including a 3D point cloud, of the target based on the 3D depth data;
acquiring a video stream of the target from a signal generated by the camera;
generating a 3D surface video based on the 3D surface contour and the video stream;
displaying the 3D surface video on the display; and
excluding occlusion events using the 3D surface contour such that an object passing between the target and at least one of the 3D scanner and the camera is not visible in the 3D surface video.

13. The method according to claim 12 , wherein the system further has a positioning device having a positioning arm with an end effector at the end of the positioning arm, the positioning device electrically coupled to the controller and at least one of the 3D scanner and the camera being mountable on the end effector.

14. The method according to claim 12 , wherein the camera is selected from a group consisting of a video camera, an infrared camera, a visible light camera, and a non-visible light camera.

15. The method according to claim 12 , wherein the 3D surface video displayed on the display shows a 3D video that is dynamically rotatable about any axis.

16. The method according to claim 12 , wherein the display includes a two dimensional video display.

17. The method according to claim 12 , wherein the display includes a stereo display system.

18. The method according to claim 12 , wherein generating a 3D surface video includes coloring each point of the 3D point cloud using colour provided by the video stream.

19. The method according to claim 12 , wherein any objects having a depth that is beyond a threshold depth distance outside of the 3D surface contour are not shown in the 3D surface video.

20. The method according to claim 12 , wherein the target includes human tissue.

21. The method according to claim 12 , the method further comprising performing calibration of input devices by mapping coordinates of the input devices into a common coordinate system, wherein the input devices include at least one of the 3D scanner, the camera, and a tracking system of the medical navigation system.

22. The method according to claim 12 , wherein displaying the 3D surface video on the display includes overlaying the 3D surface video onto a corresponding portion of pre-operative images displayed on the display.

Assignees

• Sela Gal
• Chen Sean Jy Shyang
• Mak Siu Wai Jacky
• Abhari Kamyar
• Richmond Joshua Lee
• Synaptive Medical Barbados Inc

Inventors

• Sela Gal
• Chen Sean Jy-Shyang
• Mak Siu Wai Jacky
• Abhari Kamyar
• Richmond Joshua Lee

Classifications

• H04N23/57

  Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices · CPC title

• A61B2090/373

  using light, e.g. by using optical scanners · CPC title

• A61B2034/2057

  Details of tracking cameras · CPC title

• A61B2090/363

  Use of fiducial points · CPC title

• A61B2090/103

  Cranial plugs for access to brain · CPC title