System and method for providing a contour video with a 3d surface in a medical navigation system

Patent metadata

Field	Value
Publication number	US-2018125586-A1
Application number	US-201515575552-A
Country	US
Kind code	A1
Filing date	Jul 13, 2015
Priority date	Jul 13, 2015
Publication date	May 10, 2018
Grant date	—

Abstract

A medical navigation system for displaying a three dimensional (3D) surface video of a target is provided. The medical navigation system comprises a 3D imaging device, a camera, a display, and a controller electrically coupled to the 3D imaging device, the camera, and the display. The controller has a processor coupled to a memory. The controller is configured to perform calibration of input devices; acquire 3D depth data of the target from a signal generated by the 3D imaging device; construct a 3D surface contour of the target based on the 3D depth data; acquire a video stream of the target from a signal generated by the camera; generate a 3D surface video based on the 3D surface contour and the video stream; and display the 3D surface video on the display.

First claim

We claim:

1 . A medical navigation system for displaying a three dimensional (3D) surface video of a target, the medical navigation system comprising:
a 3D imaging device; a camera; a display; and a controller electrically coupled to the 3D imaging device, the camera, and the display, the controller having a processor coupled to a memory, the controller being configured to: perform calibration of input devices; acquire 3D depth data of the target from a signal generated by the 3D imaging device; construct a 3D surface contour of the target based on the 3D depth data; acquire a video stream of the target from a signal generated by the camera; generate a 3D surface video based on the 3D surface contour and the video stream; and display the 3D surface video on the display.

2 . The medical navigation system according to claim 1 , further comprising:
a positioning device having a positioning arm with an end effector at the end of the positioning arm, the positioning device electrically coupled to the controller and at least one of the 3D imaging device and the camera being mountable on the end effector.

3 . The medical navigation system according to any one of claims 1 and 2 , wherein the 3D imaging device is selected from the group consisting of a 3D surface scanner, a structured light scanner, an optical coherence tomography (OCT) scanner, a rangefinder, and a focused light beam; and the camera is selected from the group consisting of a video camera, an infrared camera, a visible light camera, and a non-visible light camera.

4 . The medical navigation system according to any one of claims 1 - 3 , wherein the 3D surface video displayed on the display shows a 3D video that is dynamically rotatable about any axis.

5 . The medical navigation system according to any one of claims 1 - 4 , wherein the display includes a two dimensional video display.

6 . The medical navigation system according to any one of claims 1 - 4 , wherein the display includes a stereo display system.

7 . The medical navigation system according to any one of claims 1 - 6 , wherein constructing a 3D surface contour includes generating a 3D point cloud of the target based on the 3D depth data.

8 . The medical navigation system according to claim 7 , wherein generating the 3D surface video includes coloring each point of the 3D point cloud using colour provided by the video stream.

9 . The medical navigation system according to any one of claims 1 - 8 , wherein the 3D surface contour is used by the controller to exclude occlusion events such that an object passing between the target and at least one of the 3D imaging device and the camera is not visible in the 3D surface video.

10 . The medical navigation system according to claim 9 , wherein objects having a depth that is a beyond a threshold distance outside of the 3D surface contour are not shown in the 3D surface video.

11 . The medical navigation system according to any one of claims 1 - 10 , wherein the target includes human tissue.

12 . The medical navigation system according to any one of claims 1 - 11 , wherein the input devices include at least one of the 3D imaging device, the camera, and a tracking system of the medical navigation system and performing the calibration of the input devices includes mapping coordinates of the input devices into a common coordinate system.

13 . The medical navigation system according to any one of claims 1 - 12 , wherein displaying the 3D surface video on the display includes overlaying the 3D surface video onto a corresponding portion of pre-operative images displayed on the display.

14 . A method for displaying a three dimensional (3D) surface video of a target in a system having a 3D imaging device, a camera, a display, and a controller electrically coupled to the 3D imaging device, the camera, and the display, the method comprising:
performing calibration of input devices; acquiring 3D depth data of the target from a signal generated by the 3D imaging device; constructing a 3D surface contour of the target based on the 3D depth data; acquiring a video stream of the target from a signal generated by the camera; generating a 3D surface video based on the 3D surface contour and the video stream; and displaying the 3D surface video on the display.

15 . The method according to claim 14 , wherein the system further has a positioning device having a positioning arm with an end effector at the end of the positioning arm, the positioning device electrically coupled to the controller and at least one of the 3D imaging device and the camera being mountable on the end effector.

16 . The method according to any one of claims 14 and 15 , wherein the 3D imaging device is selected from the group consisting of a 3D surface scanner, a structured light scanner, an optical coherence tomography (OCT) scanner, a rangefinder, and a focused light beam; and the camera is selected from the group consisting of a video camera, an infrared camera, a visible light camera, and a non-visible light camera.

17 . The method according to any one of claims 14 - 16 , wherein the 3D surface video displayed on the display shows a 3D video that is dynamically rotatable about any axis.

18 . The method according to any one of claims 14 - 17 , wherein the display includes a two dimensional video display.

19 . The method according to any one of claims 14 - 18 , wherein the display includes a stereo display system.

20 . The method according to any one of claims 14 - 19 , wherein constructing a 3D surface contour includes generating a 3D point cloud of the target based on the 3D depth data.

21 . The method according to claim 20 , wherein generating a 3D surface video includes coloring each point of the 3D point cloud using colour provided by the video stream.

22 . The method according to any one of claims 14 - 21 , wherein the method further comprises:
excluding occlusion events using the 3D surface contour such that an object passing between the target and at least one of the 3D imaging device and the camera is not visible in the 3D surface video.

23 . The method according to claim 22 , wherein objects having a depth that is beyond a threshold distance outside of the 3D surface contour are not shown in the 3D surface video.

24 . The method according to any one of claims 14 - 23 , wherein the target includes human tissue.

25 . The method according to any one of claims 14 - 24 , wherein the input devices include at least one of the 3D imaging device, the camera, and a tracking system of the medical navigation system and performing the calibration of the input devices includes mapping coordinates of the input devices into a common coordinate system.

26 . The method according to any one of claims 14 - 25 , wherein displaying the 3D surface video on the display includes overlaying the 3D surface video onto a corresponding portion of pre-operative images displayed on the display.

Assignees

• Sela Gal
• Chen Sean Jy Shyang
• Mak Siu Wai Jacky
• Abhari Kamyar
• Richmond Joshua Lee

Inventors

• Sela Gal
• Chen Sean Jy-Shyang
• Mak Siu Wai Jacky
• Abhari Kamyar
• Richmond Joshua Lee

Classifications

• H04N23/57

  Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices · CPC title

• A61B2034/2055

  Optical tracking systems · CPC title

• A61B2090/363

  Use of fiducial points · CPC title

• A61B34/20 Primary

  Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis · CPC title

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title