Remote-control manipulator system and method of operating the same

Patent metadata

Field	Value
Publication number	US-10980606-B2
Application number	US-201615755179-A
Country	US
Kind code	B2
Filing date	May 27, 2016
Priority date	Aug 25, 2015
Publication date	Apr 20, 2021
Grant date	Apr 20, 2021

Abstract

A remote-control manipulator system includes a manipulator configured to receive a manipulating instruction from an operator, a slave arm configured to perform a series of works comprised of a plurality of processes, a camera configured to image operation of the slave arm, a display device configured to display an image captured by the camera, a storage device configured to store information related to environment in a workspace as an environment model, and a control device. The control device is configured, while operating the slave arm manually or hybridly, to acquire circumference information that is information related to a circumference area of an area imaged by the camera based on the environment model stored in the storage device, and display on the display device so that the image captured by the camera and the circumference information are interlocked.

First claim

What is claimed is:

1. A remote-control manipulator system, comprising:
a manipulator configured to receive a manipulating instruction from an operator;
a slave arm, installed in a workspace and configured to perform a series of works comprised of a plurality of processes;
a camera configured to image operation of the slave arm;
a display device configured to display an image captured by the camera;
a storage device configured to store information related to environment in the workspace as an environment model; and
a control device,
wherein the control device is configured, while operating the slave arm manually or hybridly, to acquire circumference information that is information related to a circumference area of an area imaged by the camera based on the environment model stored in the storage device, and display on the display device the image captured by the camera and the circumference information in an interlocked state, and the control device acquires the circumference information when the camera is in a zoom-in or zoom-up state, and
a scale of the circumference information displayed on the display device is different from a scale of the image captured by the camera displayed on the display device when displayed in the interlocked state.

2. The remote-control manipulator system of claim 1 , wherein the display device is configured to display the image captured by the camera in a center portion and the circumference information in a circumferential edge portion.

3. The remote-control manipulator system of claim 1 , wherein the control device is configured to acquire posture information on the slave arm, identify an imaging position of the camera based on the posture information, and identify the circumference information based on the environment model.

4. The remote-control manipulator system of claim 1 , wherein the circumference information is an operating area of the slave arm.

5. The remote-control manipulator system of claim 1 , wherein,
a plurality of slave arms are installed in the workspace, and
the control device is configured to display on the display device, posture information on a slave arm adjacent to the slave arm currently manipulated by the operator, as the circumference information.

6. The remote-control manipulator system of claim 1 , wherein the control device is configured to determine whether the slave arm currently manipulated by the operator contacts equipment disposed in the workspace.

7. The remote-control manipulator system of claim 6 , wherein the control device is configured to display warning on the display device when the control device determines that the slave arm currently manipulated by the operator contacts the equipment disposed in the workspace.

8. The remote-control manipulator system of claim 1 , wherein the circumference information is displayed entirely outside of the image captured by the camera.

9. The remote-control manipulator system of claim 1 , wherein the zoom-in or zoom-up state of the camera causes the slave arm to appear in the captured image closer than an actual distance of the slave arm from a focal plane of the camera.

10. A method of operating a remote-control manipulator system including a manipulator configured to receive a manipulating instruction from an operator, a slave arm installed in a workspace and configured to perform a series of works comprised of a plurality of processes, a camera configured to image operation of the slave arm, a display device configured to display an image captured by the camera, and a storage device configured to store information related to environment in the workspace as an environment model, comprising:
(A) while the slave arm is operated manually or hybridly, identifying circumference information that is information related to a circumference area of an area imaged by the camera based on the environment model stored in the storage device, the identifying being performed when the camera is in a zoom-in or zoom-up state; and
(B) causing the display device to display the image captured by the camera and the circumference information in an interlocked state, wherein
a scale of the circumference information displayed on the display device is different from a scale of the image captured by the camera displayed on the display device when displayed in the interlocked state.

11. The method of claim 10 , wherein the display device displays the image captured by the camera in a center portion and the circumference information in a circumferential edge portion.

12. The method of claim 10 , wherein the identifying (A) includes identifying an imaging position of the camera based on posture information on the slave arm, and identifying the circumference information based on the environment model.

13. The method of claim 10 , wherein the circumference information is an operating area of the slave arm.

14. The method of claim 10 , wherein,
a plurality of slave arms are installed in the workspace,
the identifying (A) includes causing the display device to display posture information on a slave arm adjacent to the slave arm currently manipulated by the operator as the circumference information.

15. The method of claim 10 , further comprising (C) determining whether the slave arm currently manipulated by the operator contacts equipment disposed in the workspace.

16. The method of claim 15 , further comprising, when, in the determining (C), the slave arm currently manipulated by the operator is determined to contact the equipment disposed in the workspace, (D) causing the display device to display warning.

17. The method of claim 10 , wherein the circumference information is displayed entirely outside of the image captured by the camera.

18. The method of claim 10 , wherein the zoom-in or zoom-up state of the camera causes the slave arm to appear in the captured image closer than an actual distance of the slave arm from a focal plane of the camera.

Assignees

• Kawasaki Heavy Ind Ltd

Inventors

• Hashimoto Yasuhiko
• Kamon Masayuki

Classifications

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• A61B34/70

  Manipulators specially adapted for use in surgery · CPC title

• A61B34/37 Primary

  Leader-follower robots (A61B34/35 takes precedence) · CPC title

• A61B34/35

  for telesurgery · CPC title

• B25J9/1689

  Teleoperation · CPC title