Safety monitoring system for human-machine symbiosis and method using the same

Patent metadata

Field	Value
Publication number	US-9333652-B2
Application number	US-201414224476-A
Country	US
Kind code	B2
Filing date	Mar 25, 2014
Priority date	Nov 11, 2013
Publication date	May 10, 2016
Grant date	May 10, 2016

Abstract

A safety monitoring system for human-machine symbiosis is provided, including a spatial image capturing unit, an image recognition unit, a human-robot-interaction safety monitoring unit, and a process monitoring unit. The spatial image capturing unit, disposed in a working area, acquires at least two skeleton images. The image recognition unit generates at least two spatial gesture images corresponding to the at least two skeleton images, based on information of changes in position of the at least two skeleton images with respect to time. The human-robot-interaction safety monitoring unit generates a gesture distribution based on the at least two spatial gesture images and a safety distance. The process monitoring unit determines whether the gesture distribution meets a safety criterion.

First claim

What is claimed is:

1. A safety monitoring method for human-machine symbiosis, comprising:
a spatial image capturing step for acquiring at least two skeleton images;
an image recognition step for generating at least two spatial gesture images corresponding to the at least two skeleton images, based on information of changes in position of the at least two skeleton images with respect to time;
a human-robot-interaction safety monitoring step for generating a gesture distribution based on the at least two spatial gesture images and a safety distance; and
a process monitoring step for determining whether the gesture distribution meets a safety criterion;
wherein the image recognition step comprises: acquiring images for recognizing a human skeleton, a human skeleton dilation region, a robot skeleton, and a robot skeleton dilation region, to generate the at least two spatial gesture images.

2. The method according to claim 1 , wherein the human-robot-interaction safety monitoring step sends a warning signal to a robot controller when the gesture distribution does not meet the safety criterion.

3. The method according to claim 2 , further comprising a collision warning step and a trap warning step, wherein after the warning signal is sent, the collision warning step performs a collision warning instruction and the trap warning step performs a trap warning instruction.

4. The method according to claim 1 , wherein the at least two spatial gesture images is represented by a contour map, and the human-robot-interaction safety monitoring step determines the gesture distribution, based on changes in vectors normal to a frontal surface of the contour map, and the frontal surface is an overlapping region of the at least two spatial gesture images.

5. The method according to claim 4 , wherein the human-robot-interaction safety monitoring step includes:
acquiring a human skeleton contour map, a robot skeleton contour map, a human-robot-interaction contour map, and a human-robot-interaction contour gradient map;
determining, based on the human-robot-interaction contour map, whether there is a frontal surface of the human-robot-interaction contour map at a same height, and comparing the height at which the frontal surface is with a preset value; and
determining, based on the human-robot-interaction contour gradient map, whether any contour gradient vectors cross over.

6. The method according to claim 5 , wherein if the contour map has the frontal surface existing at the same height, and the height of the frontal surface is equal to the preset value, it is determined that the gesture distribution does not meet the safety criterion.

7. The method according to claim 5 , wherein the contour gradient vectors cross over, it is determined whether directional properties of velocity paths of points of the at least two spatial gesture images change or remain unchanged, so as to perform a collision warning step or a trap warning step.

8. The method according to claim 1 , wherein the image recognition step further comprises performing an image pre-processing step for acquiring the information including a position, a continuous path, a velocity vector, a velocity path, an acceleration vector, and an acceleration path, with respect to a point in a skeleton space.

9. The method according to claim 8 , wherein the human-robot-interaction safety monitoring step comprises:
determining, based on the continuous path with respect to the point of the skeleton space, whether historical paths of points of the at least two spatial gesture images cross over at a same position, and whether paths of the points of the at least two spatial gesture images cross over at a same time;
determining, based on the velocity path with respect to the point of the skeleton space, whether historical velocity paths of the points of the at least two spatial gesture images cross over, and whether directional properties of velocity paths of the points of the at least two spatial gesture images are parallel, non-parallel, or directionless; and
determining, based on the acceleration path with respect to the point of the skeleton space, whether historical acceleration paths of the points of the at least two spatial gesture images cross over, and whether acceleration paths of the points of the at least two spatial gesture images are greater than or equal to zero.

10. The method according to claim 9 , wherein if the historical paths of the points of the at least two spatial gesture images cross over at the same position, and the paths of the points of the at least two spatial gesture images cross over at the same time, it is determined that the gesture distribution does not meet the safety criterion.

11. The method according to claim 9 , wherein if the historical velocity paths of the points of the at least two spatial gesture images cross over and the directional properties of the velocity paths of the points of the at least two spatial gesture images are non-parallel, it is determined that the gesture distribution does not meet the safety criterion.

12. The method according to claim 9 , wherein if the historical velocity paths of the points of the at least two spatial gesture images cross over, and the directional properties of the velocity paths of the points of the at least two spatial gesture images are directionless, it is further determined whether the acceleration paths of points of the at least two spatial gesture images are greater than or equal to zero.

13. The method according to claim 9 , wherein if the historical velocity paths of the points of the at least two spatial gesture images cross over, and the directional properties of the velocity paths of the points of the at least two spatial gesture images are parallel, it is further determined whether the paths of the points of the at least two spatial gesture images cross over at the same time.

14. The method according to claim 9 , wherein if the historical acceleration paths of the points of the at least two spatial gesture images cross over, it is further determined whether the acceleration paths of the points of the at least two spatial gesture images are greater than or equal to zero, so as to perform a collision warning step or a trap warning step, respectively.

Assignees

• Ind Tech Res Inst

Inventors

• Hu Jhen-Jia
• Wang Hau-Wei
• Huang Chung-Ning

Classifications

• G06V40/20 Primary

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• Y10S901/09

  Closed loop, sensor feedback controls arm movement · CPC title

• G06K9/00335

  Physics · mapped topic

• Y10S901/47

  Optical · CPC title