Configuration marker design and detection for instrument tracking

Patent metadata

Field	Value
Publication number	US-9867669-B2
Application number	US-42869109-A
Country	US
Kind code	B2
Filing date	Apr 23, 2009
Priority date	Dec 31, 2008
Publication date	Jan 16, 2018
Grant date	Jan 16, 2018

Abstract

The present disclosure relates to systems, methods, and tools for tool tracking using image-derived data from one or more tool located reference features. A method includes: directing illuminating light from a light source onto a robotic surgical tool within a patient body, wherein the tool includes a plurality of primitive features having known positions on the tool, and wherein each feature includes a spherical reflective surface; capturing stereo images of the plurality of primitive features when the tool is within the patient body, wherein the stereo images are captured by an image capture device adjacent the illumination source so that the illumination light reflected from the imaged primitive features toward the image capture device substantially aligns with spherical centers of the surfaces of the imaged primitive features, and determining a position for the tool by processing the stereo images so as to locate the spherical centers of the imaged primitive features by using the reflected light.

First claim

What is claimed is:

1. A tool tracking method for a distal end of a tool, wherein the distal end includes three primitive features having known positions on the tool, wherein each primitive feature includes a spherical reflective surface, wherein the three primitive features include a natural feature of the tool and an artificial feature of the tool, wherein the natural feature includes one of an end of a bolt and an end of a hinge of the tool and is a component of the tool that is provided for either construction or operation of the tool, and wherein the artificial feature is included on the tool to provide the spherical reflective surface, the method comprising:
receiving stereo image data of the three primitive features when the distal end of the tool is within a patient body, wherein the stereo image data captures a bright spot for each of the three primitive features which has been generated by illuminating the primitive feature, the bright spots being substantially aligned with spherical centers of the spherical reflective surfaces of the three primitive features;
generating an estimated tool state for the tool by using at least one prior tool state from the received stereo image data of the tool for a prior time or joint data from a robotic actuation system effecting movement of the tool; and
determining a pose of the tool by processing information of the stereo image data so as to locate the spherical centers by using the bright spots and based on the estimated tool state.

2. A method as in claim 1 , wherein a light used for illuminating the three primitive features includes a visible light spectrum.

3. A method as in claim 1 , wherein determining the pose of the tool by processing information of the stereo image data is accomplished so as to identify the three primitive features by using specular reflected light.

4. A method as in claim 1 , further comprising processing said stereo image data so as to determine three-dimensional positional data for the spherical centers.

5. A method as in claim 4 , wherein a constellation algorithm is used to identify a pattern of the primitive features in the stereo image data.

6. A method as in claim 5 , further comprising using the estimated tool state in the constellation algorithm.

7. A method as in claim 4 , further comprising:
capturing stereo image data for a plurality of time points; generating an estimated tool state for the plurality of time points; and
rejecting any incompatible pattern detection by using a robust estimation technique.

8. A method as in claim 7 , wherein the robust estimation technique used is a Random Sample Consensus (RANSAC) technique.

9. A method as in claim 4 , wherein a model based image signature is used in the identification of at least one primitive feature in said stereo image data.

10. A method as in claim 1 , further comprising
using the estimated tool state to reject an incompatible pattern detection.

11. A medical system, comprising:
a tool having a distal end that is insertable into a patient body, wherein the distal end has three primitive features having known positions on the tool, wherein each primitive feature comprises a spherical reflective surface, wherein the three primitive features include a natural feature of the tool and an artificial feature of the tool, wherein the natural feature includes one of an end of a bolt and an end of a hinge of the tool and is a component of the tool that is provided for either construction or operation of the tool, and wherein the artificial feature is included on the tool to a spherical reflective surface;
a light source insertable into the patient body so as to be oriented to direct light toward the distal end of the tool when the distal end is within the patient body;
a stereo image capture device insertable into the patient body so that the stereo image capture device captures stereo images of bright spots generated by the light source directing light on the three primitive features, the bright spots substantially aligned with spherical centers of the spherical surfaces; and
a processor coupled to the image capture device and configured to determine a pose of the tool by processing the stereo images so as to locate the spherical centers by using the bright spots and based on an estimated pose of the tool generated by using at least one prior pose of the tool from stereo image data of the tool for a prior time or joint data from a robotic actuation system effecting movement of the tool.

12. A system as in claim 11 , further comprising a non-transitory tangible medium comprising machine-readable instructions executable by said processor for processing said stereo images.

13. A system as in claim 11 , wherein the bright spots result from specular reflected light.

14. A system as in claim 11 , wherein the processor is configured so as to determine three-dimensional positional data for the spherical centers by processing the stereo images.

15. A system as in claim 14 , wherein the stereo image capture device comprises a stereoscopic endoscope.

16. A system as in claim 11 , wherein the spherical reflective surface comprises a convex or concave surface.

Assignees

• Zhao Tao
• Zhao Wenyi
• Nowlin William C
• Intuitive Surgical Operations

Inventors

• Zhao Tao
• Zhao Wenyi
• Nowlin William C

Classifications

• A61B34/37

  Leader-follower robots (A61B34/35 takes precedence) · CPC title

• A61B2090/371

  with simultaneous use of two cameras · CPC title

• A61B2034/102

  Modelling of surgical devices, implants or prosthesis · CPC title

• A61B2090/3983

  Reference marker arrangements for use with image guided surgery · CPC title

• A61B2034/2065

  Tracking using image or pattern recognition · CPC title