Apparatus and method for picking up article disposed in three-dimensional space using robot

Patent metadata

Field	Value
Publication number	US-9503704-B2
Application number	US-201414532680-A
Country	US
Kind code	B2
Filing date	Nov 4, 2014
Priority date	Nov 5, 2013
Publication date	Nov 22, 2016
Grant date	Nov 22, 2016

Abstract

An article pickup device configured so as to select a first and second three-dimensional points present in the vicinity of each other based on position information of the plurality of three-dimensional points acquired by a three-dimensional measurement instrument and image data acquired by a camera, acquire an image gradient information in a partial image region including points on an image corresponding to these three-dimensional points, judge whether the first and second three-dimensional points are present on the same article based on a position information of the three-dimensional points and the image gradient information, and add the first and second three-dimensional points to the same connected set when it is judged that the first and second three-dimensional points are present on the same article.

First claim

The invention claimed is:

1. An article pickup device, comprising:
a robot including a hand capable of holding an article;
a three-dimensional measurement instrument measuring surface positions of a plurality of articles disposed in a three-dimensional space to acquire a position information of a plurality of three-dimensional points;
a camera imaging an area including the plurality of articles to acquire image data;
a connected set calculation part determining a connected set made by connecting three-dimensional points present in the vicinity of each other among the plurality of three-dimensional points, based on the position information of the plurality of three-dimensional points acquired by the three-dimensional measurement instrument and the image data acquired by the camera;
an article identification part identifying a position and posture of the article, based on the position information of the three-dimensional points belonging to the connected set;
a hand position posture calculation part determining a hand position posture as a position and posture of the hand capable of picking up the article identified by the article identification part; and
a robot control part controlling the robot so as to move the hand to the hand position posture determined by the hand position posture calculation part to pick up the article,
wherein the connected set calculation part comprises:
a three-dimensional point selection part selecting a first three-dimensional point and a second three-dimensional point present in the vicinity of each other among the plurality of three-dimensional points acquired by the three-dimensional measurement instrument;
a gradient information acquisition part acquiring an image gradient information indicating a gradient state of a surface of the article in a partial image region including points on an image corresponding to respective the first three-dimensional point and the second three-dimensional point, based on the image data acquired by the camera; and
a judgment part judging whether the first three-dimensional point and the second three-dimensional point are present on the same article, based on a position information of the first three-dimensional point, a position information of the second three-dimensional point, and the image gradient information acquired by the gradient information acquisition part, and
wherein the connected set calculation part prepares a list including one three-dimensional point, sets the one three-dimensional point as a reference three-dimensional point, selects one or more three-dimensional points in the vicinity of the reference three-dimensional point by the three-dimensional point selection part, judges whether selected three-dimensional points and the reference three-dimensional point are present on the same article by the judgment part, and performs a control in which the three-dimensional point which is present on the same article is included in the list,
the connected set calculation part sets a three-dimensional point, which is not selected as the reference three-dimensional point among the three-dimensional points included in the list, as the reference three-dimensional point, selects one or more three-dimensional points in the vicinity of the reference three-dimensional point by the three-dimensional point selection part, judges whether selected three-dimensional points and the reference three-dimensional point are present on the same article by the judgment part, and performs a control in which the three-dimensional point which is present on the same article is added in the list, and
the connected set calculation part repeats the control in which the three-dimensional point is added until all three-dimensional points included in the list are set as the reference three-dimensional point, determines all three-dimensional points included in the final list as the connected set.

2. The article pickup device according to claim 1 , wherein
the gradient information acquisition part acquires points in the image corresponding to the first three-dimensional point and the second three-dimensional point respectively, as a first two-dimensional point and a second two-dimensional point specifying the partial image region, and
the judgment part judges that the first three-dimensional point and the second three-dimensional point are present on the same article, when a distance between the first three-dimensional point and the second three-dimensional point is equal to or less than a predetermined value and a change amount of contrast of the image in the partial image region is equal to or less than a predetermined value.

3. The article pickup device according to claim 2 , wherein
the gradient information acquisition part acquires the first two-dimensional point and the second two-dimensional point based on calibration data of the camera.

4. The article pickup device according to claim 1 , wherein
the article identification part determines a representative position posture as a position and posture representing the connected set, based on the position information of the three-dimensional points belonging to the connected set, and
the hand position posture calculation part determines the hand position posture corresponding to the representative position posture.

5. The article pickup device according to claim 4 , wherein
the article identification part determines the representative position posture by combining a gravity center position of all the three-dimensional points belonging to the connected set and a predetermined posture in the gravity center position.

6. The article pickup device according to claim 4 , wherein
the article identification part obtains the representative position posture by combining a center position of a circumscribed rectangle including all the three-dimensional points belonging to the connected set and a predetermined posture.

7. The article pickup device according to claim 1 , wherein
the three-dimensional measurement instrument includes the camera.

8. An article pickup method for picking up an article disposed in a three-dimensional space, using a robot including a hand capable of holding the article, the method comprising:
measuring, by a three-dimensional measurement instrument, surface positions of a plurality of articles disposed in the three-dimensional space to acquire position information of a plurality of three-dimensional points;
imaging, by a camera, an area including the plurality of articles to acquire image data;
determining a connected set made by connecting the three-dimensional points present in the vicinity of each other among the plurality of three-dimensional points, based on the position information of the plurality of three-dimensional points acquired by the three-dimensional measurement instrument and the image data acquired by the camera;
identifying a position and posture of the article, based on the position information of the three-dimensional points belonging to the connected set;
determining a hand position posture as a position and posture of the hand capable of picking up the article, the position and posture of which are identified; and
controlling the robot so as to move the hand to the hand position posture to pick up the article,
wherein the determining the connected set includes:
selecting a first three-dimensional point and a second three-dimensional point present in the vicinity of each other among the plurality of three-dimensional points acquired by the three-dimensional measurement instrument;
acquiring image gradient information indicating a gradient state of a surface of the article in a partial image region including points on an image corresponding to respective first three-dimensional point and second three-

Assignees

• Fanuc Corp

Inventors

• Ando Toshiyuki

Classifications

• G06T7/0042

  Physics · mapped topic

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• H04N13/0203 Primary

  Electricity · mapped topic

• H04N13/0246

  Electricity · mapped topic