Robotic surgical systems with user engagement monitoring

Patent metadata

Field	Value
Publication number	US-12102403-B2
Application number	US-201917777761-A
Country	US
Kind code	B2
Filing date	Dec 17, 2019
Priority date	Feb 2, 2018
Publication date	Oct 1, 2024
Grant date	Oct 1, 2024

Abstract

A robotic surgical system with user engagement monitoring includes a surgeon console having a hand detection system and a tracking device including an image capture device configured to capture an image of a user position reference point, wherein information from the hand detection system and the tracking device are combined to control operation of the robotic surgical system.

First claim

What is claimed is:

1. A robotic surgical system with user engagement monitoring, comprising:
a robot assembly including a robotic arm coupled to a surgical instrument;
a surgeon console including:
a handle assembly communicatively coupled to at least one of the robot assembly, the robotic arm, or the surgical instrument, the handle assembly including a body portion having a proximal end portion and a distal end portion, the body portion including a first actuator movable between an open position and a closed position;
a hand detection system including a first sensor disposed within the first actuator of the handle assembly for detecting finger presence on the first actuator, a second sensor disposed on the proximal end portion of the handle assembly for detecting palm presence about the proximal end portion, and a third sensor disposed within the body portion of the handle assembly for detecting position of the first actuator relative to the body portion;
a display device; and
a processing unit electrically coupled to the first, second, and third sensors for receiving and processing data from the first, second, and third sensors;

a tracking device including an image capture device configured to capture an image of a user position reference point; and
a computing device,
wherein at least one of the surgeon console, the hand detection system or the tracking device is configured to:
compute, based on the captured image, a position of the user position reference point relative to the display device,
determine whether a user is engaged with or disengaged from the surgeon console based on the computed position,
determine whether a hand of the user is engaged with or disengaged from at least one of the first, second or third sensors of the hand detection system, and
in response to a determination that the user is disengaged from the surgeon console or the hand of the user is disengaged from at least one of the first, second or third sensors of the hand detection system, cause the robotic surgical system to operate in a safe mode,

wherein at least one of the surgeon console, the hand detection system or the tracking device is further configured to, at a time when the robotic surgical system operates in the safe mode:
restrict movement of the handle assembly from a previous position of the handle assembly, and
transmit, to the computing device, instructions to restrict movement of at least one of the robot assembly, the robotic arm, or the surgical instrument;
wherein the computing device is configured to:
receive the instructions, and
transmit the instructions to the at least one of the robot assembly, the robotic arm, or the surgical instrument; and

wherein at least one of the robotic arm, the robot assembly, or the surgical instrument is configured to:
receive the instructions, and
restrict movement of at least one of the robot assembly, the robotic arm, or the surgical instrument in response to the instructions.

2. The robotic surgical system according to claim 1 , wherein at least one of the surgeon console, the hand detection system or the tracking device is further configured to compute the position of the user position reference point by generating location data corresponding to at least one of the position, or an orientation, of the user position reference point, within a three-dimensional coordinate space, relative to the display device.

3. The robotic surgical system according to claim 1 , wherein at least one of the surgeon console, the hand detection system or the tracking device is further configured to, at a time when the robotic surgical system operates in the safe mode:
in response to a determination that the user is engaged with the surgeon console by at least one of the tracking device or the hand detection system, cause the robotic surgical system to exit the safe mode after an elapsing of a threshold amount of time after the determination that the user is engaged.

4. The robotic surgical system according to claim 1 , wherein at least one of the surgeon console, the hand detection system or the tracking device is further configured to, at a time when the robotic surgical system operates in the safe mode:
prevent a movement of the handle assembly from causing a corresponding movement of the robotic arm communicatively coupled to the handle assembly.

5. The robotic surgical system according to claim 1 , wherein at least one of the surgeon console, the hand detection system or the tracking device is further configured to:
detect an amount of movement of the handle assembly;
determine, based on the amount of movement of the handle assembly, an amount of movement of at least one of the robot assembly, the robotic arm, or the surgical instrument to be caused in response to the movement of the handle assembly; and
cause at least one of the robot assembly, the robotic arm, or the surgical instrument to move by the determined amount of movement,
wherein, at a time when the robotic surgical system operates in the safe mode, the determination of the amount of movement of at least one of the robot assembly, the robotic arm, or the surgical instrument to be caused includes applying a downward scaling factor.

6. The robotic surgical system according to claim 5 , wherein at least one of the surgeon console, the hand detection system or the tracking device is further configured to:
compute a velocity of a movement of the handle assembly; and
modify the downward scaling factor based on the velocity.

7. A robotic surgical system with user engagement monitoring, comprising:
a robot assembly including a robotic arm coupled to a surgical instrument;
a surgeon console including:
a handle assembly communicatively coupled to at least one of the robot assembly, the robotic arm, or the surgical instrument, the handle assembly including a body portion having a proximal end portion and a distal end portion, the body portion including a first actuator movable between an open position and a closed position;
a hand detection system including a first sensor disposed within the first actuator of the handle assembly for detecting finger presence on the first actuator, a second sensor disposed on the proximal end portion of the handle assembly for detecting palm presence about the proximal end portion, a third sensor disposed within the body portion of the handle assembly for detecting position of the first actuator relative to the body portion;
a display device; and
a processing unit electrically coupled to the first, second, and third sensors for receiving and processing data from the first, second, and third sensors;

a tracking device including an image capture device configured to capture an image of a user position reference point,
wherein at least one of the surgeon console, the hand detection system or the tracking device is configured to:
compute, based on the captured image, a position of the user position reference point relative to the display device,
determine whether a user is engaged with or disengaged from the surgeon console based on the computed position,
determine whether a hand of the user is engaged with or disengaged from at least one of the first, second or third sensors of the hand detection system, and
in response to a determination that the user is disengaged from the surgeon console or the hand of the user is disengaged from at least one of the first, second or third sensors of the hand detection system, cause the robotic surgical system to operate in a safe mode; and

a plurality of eyewear each including a discrete plurality of markers, wherein a first user position reference point includes first data from a first plurality of markers of first eyewear corresponding to the user, and a second user positioning refe

Assignees

• Covidien Lp
• Coviden Lp

Inventors

• Peine William J
• Levine Steven J
• Dvornik Albert
• Raju Mantena V
• Chen Chen

Classifications

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• A61B90/06

  Measuring instruments not otherwise provided for · CPC title

• A61B34/74

  Manipulators with manual electric input means · CPC title

• A61B2034/305

  Details of wrist mechanisms at distal ends of robotic arms · CPC title

• A61B2034/2046

  Tracking techniques · CPC title