Estimation of a position and orientation of a frame used in controlling movement of a tool

Patent metadata

Field	Value
Publication number	US-2016206387-A1
Application number	US-201614997257-A
Country	US
Kind code	A1
Filing date	Jan 15, 2016
Priority date	May 13, 2011
Publication date	Jul 21, 2016
Grant date	—

Abstract

A robotic system includes a camera having an image frame whose position and orientation relative to a fixed frame is determinable through one or more image frame transforms, a tool disposed within a field of view of the camera and having a tool frame whose position and orientation relative to the fixed frame is determinable through one or more tool frame transforms, and at least one processor programmed to identify pose indicating points of the tool from one or more camera captured images, determine an estimated transform for an unknown one of the image and tool frame transforms using the identified pose indicating points and known ones of the image and tool frame transforms, update a master-to-tool transform using the estimated and known ones of the image and tool frame transforms, and command movement of the tool in response to movement of a master using the updated master-to-tool transform.

First claim

1 . (canceled)

2 . A method implemented in a system comprising a master control device having a master frame, an image-capturing system having an image frame, and a tool disposed within a field of view of the image-capturing system and having a tool frame, the method comprising:
specifying a sequence of frames from the image frame to the tool frame, the sequence of frames including the image frame, the tool frame, and one or more additional frames between the image frame and the tool frame; specifying a plurality of frame transforms, each frame transform of the plurality of frame transforms corresponding to a pair of adjacent frames of the sequence of frames, the plurality of frame transforms including one or more known frame transforms and an unknown frame transform between a first frame and a second frame of the sequence of frames; identifying pose-indicating points of the tool from information in one or more images captured by the image-capturing system; determining an image-to-tool frame relationship between the image frame and the tool frame from the pose-indicating points; determining an estimated frame transform for the unknown frame transform from the one or more known frame transforms and the image-to-tool frame relationship, the estimated frame transform being characterized by a chain of frame relationships between the first frame and the second frame with each of the one or more known frame transforms relating its corresponding pair of frames to one another and with the image-to-tool frame relationship relating the image frame to the tool frame; determining a master-to-tool transform by using the estimated frame transform and the one or more known frame transforms to relate the master frame to the tool frame; and moving the tool by using the master-to-tool transform to transform an input movement at the master control device to an output movement for the tool.

3 . The method of claim 2 , wherein the sequence of frames includes a first sequence of frames from the image frame to a fixed frame and a second sequence of frames from the fixed frame to the tool frame.

4 . The method of claim 2 , wherein
the system further comprises a display that is viewable by an operator, an eye frame being defined at the eyes of the operator, and a vision system configured to display information of images captured by the image-capturing system in the display; and the determining of the master-to-tool transform comprises: adjusting an offset that aligns a master-to-eye transform with an tool-to-image transform, the master-to-eye transform transforming points in the master frame to corresponding points in the eye frame, and the tool-to-image transform transforming points in the tool frame to corresponding points in the image frame.

5 . The method of claim 2 , wherein the tool moves relatively to a pivot point having a pivot-point frame, the tool includes an end effector that has an end-effector frame, and the unknown frame transform transforms points in the pivot-point frame to corresponding points in the end-effector frame.

6 . The method of claim 2 , wherein the image-capturing system is tethered to a mechanical element having a mechanical-element frame, and the unknown frame transform transforms points in the mechanical-element frame to corresponding points in the image frame.

7 . The method of claim 2 , wherein
the system further comprises a first structure having a first robotic arm coupled to the image-capturing system, the first structure having a first structure frame, and a second structure having a second robotic arm coupled to the tool, the second structure having a second structure frame; and one of the first and second structure frames is a fixed frame, and the unknown frame transform transforms points in the other of the first and second structure frames to corresponding points in the fixed frame.

8 . The method of claim 2 , wherein
the system further comprises a first structure having a first robotic arm coupled to the image-capturing system, the first structure having a first structure frame, and a second structure having a second robotic arm coupled to the tool, the second structure having a second structure frame; and at least one of the first and second structures is a movable structure.

9 . The method of claim 2 , further comprising:
determining one of the known frame transforms by using a sensor positioned along a kinematic chain extending to one of the image-capturing system or the tool.

10 . A system comprising:
a master control device having a master frame; an image-capturing system having an image frame; a tool disposed within a field of view of the image-capturing system and having a tool frame, a transform-determining device including at least one processor configured to perform operations including:
specifying a sequence of frames from the image frame to the tool frame, the sequence of frames including the image frame, the tool frame, and one or more additional frames between the image frame and the tool frame;
specifying a plurality of frame transforms, each frame transform of the plurality of frame transforms corresponding to a pair of adjacent frames of the sequence of frames, the plurality of frame transforms including one or more known frame transforms and an unknown frame transform between a first frame and a second frame of the sequence of frames;
identifying pose-indicating points of the tool from information in one or more images captured by the image-capturing system;
determining an image-to-tool frame relationship between the image frame and the tool frame from the pose-indicating points;
determining an estimated frame transform for the unknown frame transform from the one or more known frame transforms and the image-to-tool frame relationship, the estimated frame transform being characterized by a chain of frame relationships between the first frame and the second frame with each of the one or more known frame transforms relating its corresponding pair of frames to one another and with the image-to-tool frame relationship relating the image frame to the tool frame; and
determining a master-to-tool transform by using the estimated frame transform and the one or more known frame transforms to relate the master frame to the tool frame; and
a tool-moving device configured to move the tool by using the master-to-tool transform to transform an input movement at the master control device to an output movement for the tool.

11 . The system of claim 10 , wherein the sequence of frames includes a first sequence of frames from the image frame to a fixed frame and a second sequence of frames from the fixed frame to the tool frame.

12 . The system of claim 10 , further comprising:
a display that is viewable by an operator, an eye frame being defined at the eyes of the operator; and a vision system configured to display information of images captured by the image-capturing system in the display, wherein the determining of the master-to-tool transform comprises: adjusting an offset that aligns a master-to-eye transform with an tool-to-image transform, the master-to-eye transform transforming points in the master frame to corresponding points in the eye frame, and the tool-to-image transform transforming points in the tool frame to corresponding points in the image frame.

13 . The system of claim 10 , wherein the tool moves relatively to a pivot point having a pivot-point frame, the tool includes an end effector that has an end-effector frame, and the unknown frame transform transforms points in the pivot-point frame to corresponding points in the end-effector frame.

14 . The system of

Assignees

• Intuitive Surgical Operations

Inventors

• Zhao Tao
• Prisco Giuseppe Maria
• Steger John Ryan
• Larkin David Q

Classifications

• B25J9/1689

  Teleoperation · CPC title

• A61B90/361

  Image-producing devices, e.g. surgical cameras · CPC title

• A61B34/30 Primary

  Surgical robots · CPC title

• A61B34/25

  User interfaces for surgical systems · CPC title

• A61B2034/2065

  Tracking using image or pattern recognition · CPC title