Information processing apparatus, control method, robot system, and storage medium

Patent metadata

Field	Value
Publication number	US-11285603-B2
Application number	US-201916386140-A
Country	US
Kind code	B2
Filing date	Apr 16, 2019
Priority date	Apr 27, 2018
Publication date	Mar 29, 2022
Grant date	Mar 29, 2022

Abstract

The information processing apparatus includes an estimation unit to estimate information indicating a holding success possibility from an image of a plurality of the target objects by using a pre-trained model that estimates the information indicating the holding success possibility in at least one or more partial regions, a determination unit to determine a holding region for the robot to hold the target object among the partial regions based on the information indicating the holding success possibility, and a control unit to move the robot based on the holding region and cause the robot to perform a holding operation on the target object. In a case where the holding operation performed on the target object by the robot is failed, the determination unit determines a partial region, among the partial regions, satisfying a predetermined condition as a next holding region based on the information indicating the holding success possibility.

First claim

What is claimed is:

1. An information processing apparatus comprising:
one or more processors, wherein the one or more processors function as:
an estimation unit configured to estimate a holding success possibility from an image of a plurality of target objects by using a pre-trained model that estimates the holding success possibility in at least one or more positions;
a determination unit configured to determine position satisfying a predetermined condition in a holding range in which an end effector of a robot acts on a target object, of the plurality of target objects, as a holding position for holding the target object, sequentially from among the at least one or more positions, based on the holding success possibility; and
a control unit configured to move the robot based on the holding position determined by the determination unit and cause the robot to perform a holding operation on the target object,
wherein the determination unit is configured to, in a case where the holding operation performed on the target object by the robot is failed, determine a next holding position from a peripheral range of the holding position in the holding range in the holding operation in a case where there is a position other than the holding position satisfying the predetermined condition in the holding range, and determine the next holding position from positions outside the holding range satisfying the predetermined condition in a case where there is not a position satisfying the predetermined condition in the holding range, based on the holding success possibility.

2. The information processing apparatus according to claim 1 ,
wherein the robot is configured to perform the holding operation by suction of the target object, and
wherein the determination unit is configured to, in a case where the robot fails to suction the target object, determine position satisfying the predetermined condition as the next holding position based on the holding success possibility.

3. The information processing apparatus according to claim 1 , wherein the pre-trained model is a neural network into which an image of the plurality of the target objects is input and is trained to output image information about the image, the image information indicating at least one or more positions as a holding position(s).

4. The information processing apparatus according to claim 3 , wherein the pre-trained model is a neural network trained with training data obtained by a person giving a true value to a captured image of a group of the target objects disorderly arranged, the true value indicating a position where the target object is successfully held.

5. The information processing apparatus according to claim 4 , wherein the determination unit is configured to prevent a predetermined position from being determined as the holding position even in a case where a holding success probability based on the holding success possibility is higher than a predetermined threshold, the predetermined region being set based on a shape of a container containing the target object.

6. The information processing apparatus according to claim 1 , wherein the predetermined condition is that a holding success probability based on the information indicating the holding success possibility be higher than a predetermined threshold.

7. The information processing apparatus according to claim 1 ,
wherein the determination unit is configured to determine a plurality of the holding positions based on the holding success possibility, and
wherein the predetermined condition is that a position to be determined as the next holding position be different from the previous holding positions determined by the determination unit.

8. The information processing apparatus according to claim 1 , further comprising an obtaining unit configured to obtain force information indicating external force acting when the robot makes contact with the target object,
wherein the control unit is configured to continue moving the robot until the force information satisfies a second predetermined condition.

9. The information processing apparatus according to claim 1 , further comprising an obtaining unit configured to obtain proximity information indicating a proximity distance between the robot and the target object,
wherein the control unit is configured to perform control to cause the robot to approach the target object in a normal direction based on the proximity information.

10. The information processing apparatus according to claim 1 ,
wherein the determination unit is configured to determine an order of priority of a plurality of positions to be candidates for the holding position based on the holding success possibility, and
wherein the determination unit is configured to, in a case where the holding operation performed on the target object by the robot is failed, determine the holding position based on the order of priority.

11. The information processing apparatus according to claim 10 , wherein the determination unit is configured to determine the order of priority in such a manner that the order of priority decreases toward a corner of a container containing the target object.

12. The information processing apparatus according to claim 1 ,
wherein the robot includes a suction device configured to suction the target object, and
wherein the information processing apparatus further comprises a second determination unit configured to, when the robot performs the holding operation, determine whether the robot succeeds or fails in holding the target object based on a measurement result obtained by a degree of vacuum in a space formed between the robot and the target object being measured.

13. The information processing apparatus according to claim 12 , wherein the second determination unit is configured to, in a case where the measurement result of the suction device reaches a predetermined value, determine that the holding is successful, and in a case where the measurement result does not reach the predetermined value, determine that the holding is failed.

14. The information processing apparatus according to claim 1 , further comprising a storage unit configured to, in a case where it is determined that the holding of the target object performed by the robot is failed, store the holding position at that time,
wherein the determination unit is configured not to determine the previous holding position(s) determined by the determination unit as a new holding position before a state of the target object changes.

15. The information processing apparatus according to claim 1 , wherein the image includes distance information between the target object and an imaging device.

16. The information processing apparatus according to claim 1 ,
wherein the robot includes an imaging device configured to capture the target object and a surrounding object, and
wherein the control unit is configured to control movement of an arm of the robot.

17. The information processing apparatus according to claim 16 , wherein the control unit is configured to control the robot in such a manner that a moving distance to the holding position determined by the determination unit is shortened and that the robot takes a trajectory that maximizes a moving speed of the robot.

18. The information processing apparatus according to claim 1 , wherein the information indicating the holding success possibility includes a distribution indicating a probability of succeeding in holding the target object with respect to each pixel of the image.

19. The infor

Assignees

• Canon Kk

Inventors

• Yamada Daisuke
• Suzuki Masahiro
• Higo Tomoaki
• Hirota Yuichiro

Classifications

• G05B2219/37563

  Ccd, tv camera · CPC title

• B25J9/1697 Primary

  Vision controlled systems · CPC title

• G05B2219/40053

  Pick 3-D object from pile of objects · CPC title

• G05B2219/37572

  Camera, tv, vision · CPC title

• B25J9/1661

  characterised by task planning, object-oriented languages · CPC title