Force-feedback gloves for a surgical robotic system

Patent metadata

Field	Value
Publication number	US-11457986-B2
Application number	US-202017039705-A
Country	US
Kind code	B2
Filing date	Sep 30, 2020
Priority date	Sep 30, 2020
Publication date	Oct 4, 2022
Grant date	Oct 4, 2022

Abstract

A surgical robotic system that includes a robotic arm, glove configured to be worn on a hand of a user and including a force-feedback mechanism, a tracking device, a processor, and memory. The memory includes instructions which when executed by the processor causes the system to determine that the user is performing a hand gesture with the glove to grasp a virtual user input device (UID) based on the tracking device, and in response to the user grasping the virtual UID, apply, via the force-feedback mechanism, a force upon the glove that corresponds to a physical representation of the virtual UID, and engage the robotic arm to be controlled by the virtual UID.

First claim

What is claimed is:

1. A surgical robotic system comprising:
a robotic arm;
a glove configured to be worn on a hand of a user and including a force-feedback mechanism;
a tracking device;
a processor; and
memory having instructions which when executed by the processor causes the system to
determine that the user is performing a hand gesture with the glove to grasp a virtual user input device (UID) based on the tracking device;
in response to the user grasping the virtual UID,
apply, via the force-feedback mechanism, a force upon the glove that corresponds to a physical representation of the virtual UID; and
engage the robotic arm to be controlled by the virtual UID.

2. The surgical robotic system of claim 1 , wherein the hand gesture is a first hand gesture, wherein the memory has further instructions to
determine that the user is performing a second hand gesture with the glove to release the virtual UID based on the tracking device; and
in response to the user releasing the virtual UID, disengage the robotic arm from being controlled by the virtual UID.

3. The surgical robotic system of claim 1 , the hand gesture is a first hand gesture, wherein the memory has further instructions to
determine that the user is performing a second hand gesture with the glove to manipulate the virtual UID based on the tracking device; and
control the robotic arm according to the second hand gesture.

4. The surgical robotic system of claim 3 , wherein the second hand gesture is at least one of a squeezing motion that represents squeezing the virtual UID or a hand movement that represents moving the virtual UID.

5. The surgical robotic system of claim 1 , wherein the glove includes a haptic feedback mechanism, wherein the memory has further instructions to apply, via the haptic feedback mechanism and upon the glove grasping the virtual UID, a plurality of forces that represent a surface of the physical representation of the virtual UID.

6. The surgical robotic system of claim 1 , wherein the force is a first force and the virtual UID comprises an inner portion that is nested inside an outer portion, wherein the memory has further instructions to
determine that a portion of the glove is moving inside the outer portion; and
in response to determining that the portion of the glove is moving inside the outer portion, apply, via the force feedback mechanism, a second force upon the portion of the glove.

7. The surgical robotic system of claim 6 , wherein the first force is stronger than the second force.

8. The surgical robotic system of claim 6 , wherein in response to the user grasping the inner portion the force feedback mechanism is arranged to apply the first force that corresponds to the physical representation of the virtual UID.

9. The surgical robotic system of claim 8 , wherein the hand gesture is a first hand gesture, wherein the memory has further instructions to
determine that the user is performing a second hand gesture with the glove to cause another portion of the glove that is inside the outer portion to extend outside and away from the outer portion based on the tracking device, while continuing to grasp the inner portion; and
in response to determining that the user is performing the second hand gesture,
apply, via the force-feedback mechanism, the second force upon the other portion of the glove until it exits the outer portion, and
disengage the robotic arm from being controlled by the virtual UID upon the other portion of the glove exiting the outer portion.

10. The surgical robotic system of claim 9 , wherein the other portion of the glove is at least one finger.

11. The surgical robotic system of claim 1 , wherein the virtual UID has a location within a physical space that is proximate to a palmar side of the glove and is stationary with respect to the glove such that the virtual UID mirrors movements of the glove within the physical space.

12. The surgical robotic system of claim 1 , wherein the robotic arm is a first robotic arm, wherein the surgical robotic system comprises a second robotic arm, wherein the instructions to determine that the user is grasping the virtual UID comprises instructions to determine a first finger and the thumb are grasping the virtual UID, wherein the memory has further instructions to
determine that a second finger and the thumb are grasping the virtual UID based on the tracking device; and
in response to the user grasping the virtual UID with the second finger, engage the second robotic arm to be controlled by the virtual UID.

13. A method performed by a programmed processor of a surgical robotic system that includes a glove configured to be worn on a hand of a user and a robotic arm, the method comprising:
determine that the user is grasping a virtual user input device (UID) with the glove;
in response to the user grasping the virtual UID,
apply, via a force-feedback mechanism of the glove, a force upon the glove that corresponds to a physical representation of the virtual UID, and
engage the robotic arm to be controlled by the virtual UID.

14. The method of claim 13 , wherein the hand gesture is a first hand gesture, wherein the method further comprises
determining that the user is performing a second hand gesture with the glove to release the virtual UID; and
in response to the user releasing the virtual UID, disengaging the robotic arm from being controlled by the virtual UID.

15. The method of claim 13 , wherein the hand gesture is a first hand gesture, wherein the method further comprises
determining that the user is performing a second hand gesture with the glove to manipulate the virtual UID; and
controlling the robotic arm according to the second hand gesture.

16. The method of claim 15 , wherein the second hand gesture is at least one of a squeezing motion that represents squeezing the virtual UID or a hand movement that represents moving the virtual UID.

17. The method of claim 13 , wherein the force is a first force and the virtual UID comprises an inner portion that is nested inside an outer portion, the method further comprises
determining that a portion of the glove is moving inside the outer portion; and
in response to determining that the portion of the glove is moving inside the outer portion, apply a second force upon the portion of the glove.

18. The method of claim 17 , wherein the first force is stronger than the second force.

19. The method of claim 17 , wherein grasping the virtual UID comprises grasping the inner portion of the virtual UID.

20. The method of claim 19 , wherein the hand gesture is a first hand gesture, wherein the method further comprises
determining that the user is performing a second hand gesture with the glove to cause another portion of the glove that is inside the outer portion to extend outside and away from the outer portion, while continuing to grasp the inner portion; and
in response to determining that the user is performing the second hand gesture,
applying the second force upon the other portion of the glove unit it exits the outer portion, and
disengaging the robotic arm from being controlled by the virtual UID upon the other portion of the glove exiting the outer portion.

Assignees

• Verb Surgical Inc

Inventors

• Xu Yiming
• Gonenc Berk
• Olson Blade

Classifications

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• A61B34/76

  Manipulators having means for providing feel, e.g. force or tactile feedback · CPC title

• G06F3/016

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• A61B34/74 Primary

  Manipulators with manual electric input means · CPC title

• A61B34/37

  Leader-follower robots (A61B34/35 takes precedence) · CPC title