Image processing method, device and non-transitory computer-readable medium

Patent metadata

Field	Value
Publication number	US-10074010-B2
Application number	US-201615211924-A
Country	US
Kind code	B2
Filing date	Jul 15, 2016
Priority date	Jul 30, 2015
Publication date	Sep 11, 2018
Grant date	Sep 11, 2018

Abstract

An image processing method in accordance with this disclosure includes steps of obtaining an object image including an operating body, extracting the operating body from the object image, and calculating an orientation of the operating body in the object image. The method further includes determining whether the operating body is of an operator that operates an apparatus based on the orientation of the operation body.

First claim

What is claimed is:

1. An image processing method, comprising:
obtaining an object image including an operating body;
extracting the operating body from the object image;
calculating, by processing circuitry, an orientation of the operating body in the object image; and
determining, by the processing circuitry, whether the operating body is of an operator that operates an apparatus based on the orientation of the operation body, wherein
the calculating the orientation includes:
retrieving contour pixels of the operating body;
computing a curvature density of a predetermined range around the contour pixels;
clustering the curvature density into a first curvature density cluster with a first curvature density cluster center and a second curvature density cluster with a second curvature density cluster center, such that the curvature density of the first curvature density cluster is higher than the one of the second curvature density cluster; and
determining, based on the first curvature density cluster center and the second curvature density cluster center, the orientation of the operating body.

2. The image processing method according to claim 1 , further comprising:
obtaining depth information of the operating body, wherein
the determining includes determining that the operating body is of the operator when the orientation of the operation body meets a first condition and the depth information meets a second condition.

3. The image processing method according to claim 1 , wherein
the calculating the orientation further includes, after the computing and before the clustering, adjusting the curvature density, and
the adjusting the curvature density includes:
increasing the curvature density when the curvature density higher than a predetermined threshold; and
decreasing the curvature density when the curvature density lower than the predetermined threshold.

4. The image processing method according to claim 3 , wherein
the predetermined threshold is a predetermined threshold value,
increasing the curvature density includes selecting a highest curvature density in an assemblage and adjusting the curvature density of the contour pixel to the highest curvature density when an average curvature density of the assemblage of the contour pixel and non-zero pixels around the contour pixel is equal or higher than the predetermined threshold value, and
decreasing the curvature density includes selecting a lowest curvature density in an assemblage and adjusting the curvature density of the contour pixel to the lowest curvature density when an average curvature density of the assemblage of the contour pixel and non-zero pixels around the contour pixel is lower than the predetermined threshold value.

5. The image processing method according to claim 1 , wherein when the object image includes a plurality of operating bodies, the method further comprises:
extracting the plurality of operating bodies from the object image;
calculating, by the processing circuitry, an orientation of each operating body of the plurality of operating bodies in the object image; and
determining, by the processing circuitry, which operating body of the plurality of operating bodies is of the operator based on the orientation of each operating body of the plurality of operating bodies.

6. An image processing device, comprising:
an obtaining device that obtains an object image including an operating body; and
processing circuitry configured to
extract the operating body from the object image;
calculate the orientation of each operating body in the object image; and
determine whether the operating body is of an operator that operates an apparatus based on the orientation of the operation body, wherein

to calculate the orientation of the operating body, the processing circuitry is configured to:
retrieve contour pixels of the operating body;
compute a curvature density of a predetermined range around the contour pixels;
cluster the curvature density into a first curvature density cluster with a first curvature density cluster center and a second curvature density cluster with a second curvature density cluster center such that the curvature density of the first curvature density cluster is higher than the one of the second curvature density cluster; and
determine, based on the first curvature density cluster center and the second curvature density cluster center, the orientation of the operating body.

7. The image processing device according to claim 6 , wherein the processing circuitry is further configured to:
obtain depth information of the operating body,
determine that the operating body is operating the apparatus when the orientation of the operation body meets a first condition and the depth information meets a second condition.

8. The image processing device according to claim 6 , wherein
to calculate the orientation, the processing circuitry is configured to adjust the curvature density after computing the curvature density and before the clustering, and
to adjust the curvature density, the processing circuitry is configured to:
adjust the curvature density of the contour pixels by increasing the curvature density when the curvature density is equal to or higher than a predetermined threshold; and
decrease the curvature density when the curvature density is lower than the predetermined threshold.

9. The image processing device according to claim 8 , wherein
the predetermined threshold is a predetermined threshold value,
to increase the curvature density, the processing circuitry is configured to select a highest curvature density in an assemblage and adjusting the curvature density of the contour pixel to the highest curvature density when an average curvature density of the assemblage of the contour pixel and non-zero pixels around the contour pixel is equal or higher than the predetermined threshold value, and
to decrease the curvature density, the processing circuitry is configured to select a lowest curvature density in an assemblage and adjusting the curvature density of the contour pixel to the lowest curvature density when an average curvature density of the assemblage of the contour pixel and non-zero pixels around the contour pixel is lower than the predetermined threshold value.

10. The image processing device according to claim 6 , wherein
the processing circuitry is configured to, when the object image includes a plurality of operating bodies:
extract the plurality of operating bodies from the object image;
calculate an orientation of each operating body of the plurality of operating bodies in the object image; and
determine which operating body of the plurality of operating bodies is of the operator based on the orientation of each operating body of the plurality of operating bodies.

11. The image processing device according to claim 6 , wherein when the object image includes a plurality of operating bodies, the processing circuitry is configured to determine which operating body of the plurality of operating bodies is of the operator based on an orientation of each operating body of the plurality of operating bodies.

12. A non-transitory computer-readable medium including computer executable instructions which, when executed by processing circuitry, cause the processing circuitry to execute a process comprising:
obtaining an object image including an operating body;
extracting the operating body from the object image;
calculating an orientation of the operating body in the object image; and
determining whether the operating body is of an operator that operates an apparatus based on the orientation of the operation body, whe

Assignees

• Chen Chao
• Liu Liyan
• Zhao Ying
• Ricoh Co Ltd

Inventors

• Chen Chao
• Liu Liyan
• Zhao Ying

Classifications

• G06V40/107 Primary

  Static hand or arm · CPC title

• G06V40/20

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title

• G06V10/46

  Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features (colour feature extraction G06V10/56) · CPC title

• G06K9/00375

  Physics · mapped topic

• G06K9/48

  Physics · mapped topic