Information processing apparatus, information processing method, and system

Patent metadata

Field	Value
Publication number	US-11541545-B2
Application number	US-201916578073-A
Country	US
Kind code	B2
Filing date	Sep 20, 2019
Priority date	Sep 27, 2018
Publication date	Jan 3, 2023
Grant date	Jan 3, 2023

Abstract

An information processing apparatus includes an acquisition unit acquiring a first image and a second image, the first image being an image of a target area in an initial state, the second image being an image of the target area where a first object conveyed from a supply area is placed, an estimation unit estimating one or more second areas in the target area, based on a feature of a first area estimated using the first image and the second image, the first area being where the first object is placed, the one or more second areas each being an area where an object in the supply area can be placed and being different from the first area. A control unit controls a robot to convey a second object different from the first object from the supply area to any of the one or more second areas.

First claim

What is claimed is:

1. An information processing apparatus for controlling a robot that conveys an object from a supply area to a target area, the information processing apparatus comprising:
an acquisition unit configured to acquire a first image and a second image, the first image being an image of the target area in an initial state, the second image being an image of the target area where a first object conveyed by a user from the supply area is placed;
an estimation unit configured to estimate one or more second areas in the target area, based on a shape feature of a first area in the target area before the first object is placed by the user estimated using the first image and the second image, the one or more second areas each being an area where an object in the supply area can be placed and being ata different position from the first area; and
a control unit configured to control the robot to convey a second object different from the first object from the supply area to any of the second areas.

2. The information processing apparatus according to claim 1 , wherein the control unit controls the robot to convey the second object to any of the second areas not having the first object.

3. The information processing apparatus according to claim 1 , wherein in a case where the second object does not remain in the supply area, the control unit stops the robot.

4. The information processing apparatus according to claim 1 , wherein in a case where the target area has none of the second areas where an object can be placed, the control unit stops the robot.

5. The information processing apparatus according to claim 1 , wherein the target area is an area having partial areas which are similar in shape to the first area and are regularly arranged.

6. The information processing apparatus according to claim 1 , wherein the control unit controls the robot to convey and place the second object in such a manner that a direction of the second object is aligned with a direction of the first object.

7. The information processing apparatus according to claim 1 , wherein the control unit controls the robot to place the second object in such a manner that a front-back direction of the second object corresponds to a front-back direction of the first object.

8. The information processing apparatus according to claim 1 , wherein based on a difference between the first image and the second image, the estimation unit estimates the location of the first area where the first object is placed.

9. The information processing apparatus according to claim 1 , wherein the feature of the first area is acquired from the first image.

10. The information processing apparatus according to claim 1 , wherein the estimation unit estimates the one or more second areas in the target area, by template matching between the feature of the first area and a feature of a portion of the first image.

11. The information processing apparatus according to claim 1 , further comprising an instruction unit configured to notify a user that the user performs an operation for conveying the first object from the supply area to the target area, after the first image is captured.

12. The information processing apparatus according to claim 11 , wherein in a case where none of the second objects remain in the supply area, the instruction unit notifies the user that the operation is completed.

13. The information processing apparatus according to claim 11 , wherein in a case where the target area has none of the second areas where an object can be placed, the instruction unit notifies the user that the operation is completed.

14. The information processing apparatus according to claim 1 ,
wherein the acquisition unit further acquires a third image obtained by image-capturing the supply area where the first object is randomly placed, and
wherein the estimation unit estimates a candidate for a holding position by template matching between the third image and the feature of the first area which is estimated from the first image and the second image and has the first object.

15. The information processing apparatus according to claim 1 , further comprising:
a display unit configured to display the second areas estimated by the estimation unit, using a predetermined method; and
a confirmation reception unit configured to receive from a user a determination of whether conveyance of the second object to the second areas is appropriate,
wherein in a case where the user determines that the conveyance is not appropriate, the acquisition unit acquires a third image captured in a state where the first object has been placed again in a fourth area in the target area, and
wherein based on the first image and the third image, the estimation unit estimates the fourth area as the second areas.

16. The information processing apparatus according to claim 15 , wherein the display unit displays on a display device an area corresponding to second areas in the first image.

17. The information processing apparatus according to claim 15 , wherein the display unit projects a predetermined image onto the second areas using a projection device.

18. The information processing apparatus according to claim 1 , wherein the acquisition unit acquires the first image being the image of the target area in the initial state in which the first object is not yet placed.

19. A non-transitory storage medium that stores a program for causing a computer to execute an information processing method for generating a control value of a robot that conveys an object from a supply area to a target area, the information processing method comprising:
acquiring a first image and a second image, the first image being an image of a target area in an initial state, the second image being an image of the target area where a first object conveyed by a user from a supply area is placed;
estimating, in the target area, one or more second areas in the target area, based on a shape feature of a first area in the target area before the first object is played by the user estimated using the first image and the second image, the one or more second areas each being an area where an object in the supply area can be placed and being ata different position from the first area; and
generating the control value with which the robot is to convey a second object different from the first object from the supply area to any of the one or more second areas.

20. An information processing method for generating a control value of a robot that conveys an object from a supply area to a target area, the information processing method comprising:
acquiring a first image and a second image, the first image being an image of a target area in an initial state, the second image being an image of the target area where a first object conveyed by a user from a supply area is placed;
estimating, in the target area, one or more second areas in the target area, based on a shape feature of a first area in the target area before the first object is played by the user estimated using the first image and the second image, the one or more second areas each being an area where an object in the supply area can be placed and being ata different position from the first area; and
generating the control value with which the robot is to convey a second object different from the first object from the supply area to any of the one or more second areas.

21. An information processing system for generating a control value use by a robot that conveys an object f

Assignees

• Canon Kk

Inventors

• Yamada Daisuke
• Higo Tomoaki
• Kobayashi Kazuhiko

Classifications

• G05B2219/40532

  Ann for vision processing · CPC title

• B25J11/00

  Manipulators not otherwise provided for · CPC title

• H04N23/695

  Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects · CPC title

• G06V40/107

  Static hand or arm · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title