Safety monitoring system for human-machine symbiosis and method using the same

Patent metadata

Field	Value
Publication number	US-2015131896-A1
Application number	US-201414224476-A
Country	US
Kind code	A1
Filing date	Mar 25, 2014
Priority date	Nov 11, 2013
Publication date	May 14, 2015
Grant date	—

Abstract

A safety monitoring system for human-machine symbiosis is provided, including a spatial image capturing unit, an image recognition unit, a human-robot-interaction safety monitoring unit, and a process monitoring unit. The spatial image capturing unit, disposed in a working area, acquires at least two skeleton images. The image recognition unit generates at least two spatial gesture images corresponding to the at least two skeleton images, based on information of changes in position of the at least two skeleton images with respect to time. The human-robot-interaction safety monitoring unit generates a gesture distribution based on the at least two spatial gesture images and a safety distance. The process monitoring unit determines whether the gesture distribution meets a safety criterion.

First claim

What is claimed is:

1 . A safety monitoring system for human-machine symbiosis, comprising:
a spatial image capturing unit, disposed in a working area, is configured to acquire at least two skeleton images; an image recognition unit is configured to generate at least two spatial gesture images corresponding to the at least two skeleton images, based on information of changes in position of the at least two skeleton images with respect to time; a human-robot-interaction safety monitoring unit is configured to generate a gesture distribution based on the at least two spatial gesture images and a safety distance; and a process monitoring unit is configured to determine whether the gesture distribution meets a safety criterion.

2 . The system according to claim 1 , wherein the human-robot-interaction safety monitoring unit sends a warning signal to a robot controller when the gesture distribution does not meet the safety criterion.

3 . The system according to claim 2 , wherein the robot controller comprises a collision warning module and a trap warning module, wherein when the robot controller receives the warning signal, the collision warning module performs an instruction to avoid collision and the trap warning module performs an instruction to escape being trapped.

4 . The system according to claim 1 , further comprising an image database, coupled to the human-robot-interaction safety monitoring unit, is configured to store an image of the gesture distribution occurred within a preset time period.

5 . The system according to claim 1 , wherein the human-robot-interaction safety monitoring unit uses a contour map representing the at least two spatial gesture images, and the human-robot-interaction safety monitoring unit determines the gesture distribution, based on changes in vectors normal to a frontal surface of the contour map, and the frontal surface is an overlapping region of the at least two spatial gesture images.

6 . The system according to claim 1 , wherein the image recognition unit generates an expanded skeleton image.

7 . The system according to claim 6 , wherein the image recognition unit recognizes gestures of the head, hand(s), arm(s) of a human and gestures of arm(s) of a robot, based on the expanded skeleton image.

8 . The system according to claim 6 , wherein the human-robot-interaction safety monitoring unit includes a working-area dynamic monitoring unit for monitoring human and robot working areas, wherein the working area dynamic monitoring unit determines a protection boundary value with respect to human and robot, based on the expanded skeleton image.

9 . The system according to claim 1 , wherein the information includes a human movement path and a robot movement path.

10 . The system according to claim 9 , wherein the human movement path includes a human position path, a human velocity path, and a human acceleration path, and the robot movement path includes a robot position path, a robot velocity path, and a robot acceleration path; and the human-robot-interaction safety monitoring unit determines whether the human movement path and the robot movement path overlap, based on the human position path, the human velocity path, the human acceleration path, the robot position path, the robot velocity path, and the robot acceleration path.

11 . The system according to claim 10 , wherein the human-robot-interaction safety monitoring unit determines that historical paths of points of the at least two spatial gesture images cross over at a same position, and further determines whether paths of the points of the at least two spatial gesture images cross over at a same time; if so, the gesture distribution does not meet the safety criterion; if not, the gesture distribution meets the safety criterion.

12 . A safety monitoring method for human-machine symbiosis, comprising:
a spatial image capturing step for acquiring at least two skeleton images; an image recognition step for generating at least two spatial gesture images corresponding to the at least two skeleton images, based on information of changes in position of the at least two skeleton images with respect to time; a human-robot-interaction safety monitoring step for generating a gesture distribution based on the at least two spatial gesture images and a safety distance; and a process monitoring step for determining whether the gesture distribution meets a safety criterion.

13 . The method according to claim 12 , wherein the human-robot-interaction safety monitoring step sends a warning signal to a robot controller when the gesture distribution does not meet the safety criterion.

14 . The method according to claim 13 , further comprising a collision warning step and a trap warning step, wherein after the warning signal is sent, the collision warning step performs a collision warning instruction and the trap warning step performs a trap warning instruction.

15 . The method according to claim 12 , wherein the image recognition step comprises: acquiring images for recognizing a human skeleton, a human skeleton dilation region, a robot skeleton, and a robot skeleton dilation region, to generate the at least two spatial gesture images.

16 . The method according to claim 15 , wherein the at least two spatial gesture images is represented by a contour map, and the human-robot-interaction safety monitoring step determines the gesture distribution, based on changes in vectors normal to a frontal surface of the contour map, and the frontal surface is an overlapping region of the at least two spatial gesture images.

17 . The method according to claim 16 , wherein the human-robot-interaction safety monitoring step includes:
acquiring a human skeleton contour map, a robot skeleton contour map, a human-robot-interaction contour map, and a human-robot-interaction contour gradient map; determining, based on the human-robot-interaction contour map, whether there is a frontal surface of the human-robot-interaction contour map at a same height, and comparing the height at which the frontal surface is with a preset value; and determining, based on the human-robot-interaction contour gradient map, whether any contour gradients vector cross over.

18 . The method according to claim 17 , wherein if the contour map has the frontal surface existing at the same height, and the height of the frontal surface is equal to the preset value, it is determined that the gesture distribution does not meet the safety criterion.

19 . The method according to claim 17 , wherein the contour gradient vector crosses over, it is determined whether directional properties of velocity paths of points of the at least two spatial gesture images change or remain unchanged, so as to perform a collision warning step or a trap warning step.

20 . The method according to claim 15 , wherein the image recognition step further comprises performing an image pre-processing step for acquiring the information including a position, a continuous path, a velocity vector, a velocity path, an acceleration vector, and an acceleration path, with respect to a point in a skeleton space.

21 . The method according to claim 20 , wherein the human-robot-interaction safety monitoring step comprises:
determining, based on the continuous path with respect to the point of the skeleton space, whether historical paths of points of the at least two spatial gesture images cross over at a same position, and whether paths of the points of the at least two spatial gesture i

Assignees

• Ind Tech Res Inst

Inventors

• Hu Jhen-Jia
• Wang Hau-Wei
• Huang Chung-Ning

Classifications

• G06V40/20 Primary

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title

• Y10S901/47

  Optical · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• Y10S901/09

  Closed loop, sensor feedback controls arm movement · CPC title

• G06K9/00335

  Physics · mapped topic