Iterative control of robot for target object

Patent metadata

Field	Value
Publication number	US-12583123-B2
Application number	US-202418407440-A
Country	US
Kind code	B2
Filing date	Jan 9, 2024
Priority date	Jul 12, 2021
Publication date	Mar 24, 2026
Grant date	Mar 24, 2026

Abstract

A robot control system includes circuitry to iteratively move a robot toward a task area in which the robot is to perform a task on a target object, by acquiring a first image of an observation area in a vicinity of the robot from an image sensor, calculating a probability that the observation area includes the task area based on the first image, extracting the task area from the first image based on the probability, controlling the robot to cause the robot to approach the task area, acquiring a second image of the observation area from the image sensor, after the robot approaches the task area, calculating a probability that the observation area includes the task area based on the second image, extracting the task area from the second image based on the probability, and controlling the robot to further approach the task area extracted from the second image.

First claim

What is claimed is:

1 . A robot control system comprising circuitry configured to:
acquire a first image of an observation area in a vicinity of a robot, from an image sensor; calculate a probability that the observation area includes a task area in which the robot is to perform a task on a target object, based on the first image, wherein the circuitry is configured to compare a configuration of an end effector of the robot with a structure of the target object included in the observation area based on the first image to calculate the probability; extract the task area from the first image, based on the probability calculated; control the robot to cause the robot to approach the task area; acquire a second image of the observation area from the image sensor, after the robot approaches the task area; calculate the probability that the observation area includes the task area, based on the second image, wherein the circuitry is configured to compare the configuration of the end effector with the structure of the target object included in the observation area based on the second image to calculate the probability; extract the task area from the second image based on the probability associated with the second image; and control the robot to further approach the task area extracted from the second image, to iteratively move the robot toward the task area.

2 . The robot control system according to claim 1 , wherein the circuitry is further configured to cause the robot to perform the task, in response to detecting that the robot has reached the task area.

3 . The robot control system according to claim 1 , wherein the image sensor is movable in response to a movement of the robot.

4 . The robot control system according to claim 1 , wherein the circuitry is configured to:
extract from the first image a plurality of task areas including the task area, associated with a plurality of probabilities; and identify a selected task area from the plurality of task areas, based on a comparison of the plurality of probabilities, wherein the task area that the robot approaches corresponds to the selected task area.

5 . The robot control system according to claim 1 , wherein the circuitry is configured to, for each of a plurality of tasks different from each other, extract the task area corresponding to the task, from the first image.

6 . The robot control system according to claim 1 , wherein the task area is further extracted based on a configuration of the robot and a current state of the observation area.

7 . The robot control system according to claim 6 ,
wherein the current state includes whether the observation area includes the target object, and wherein the task area is extracted further based on a determination that the observation area includes the target object.

8 . The robot control system according to claim 7 ,
wherein the current state further includes whether the observation area includes an obstacle different from the target object, and wherein the task area is extracted further based on a determination that the obstacle does not present an interference for the robot to perform the task on the target object.

9 . The robot control system according to claim 7 ,
wherein the configuration of the robot includes a shape of an end effector of the robot, wherein the current state further includes a shape of the target object, and wherein the task area is extracted further based on a comparison of the shape of the end effector with the shape of the target object.

10 . The robot control system according to claim 1 , wherein the task area is extracted from the first image in response to determining that the probability associated with the first image reaches a predetermined threshold.

11 . The robot control system according to claim 10 ,
wherein the circuitry is configured to calculate a distribution of probabilities in the observation area from the first image, and wherein the task area is extracted from the first image by selecting a region of the first image in which the probability reaches the threshold, based on the distribution.

12 . The robot control system according to claim 1 ,
wherein the observation area is a first observation area in a three-dimensional space in the vicinity of the robot, and wherein the circuitry is further configured to: acquire a third image of a second observation area in the three-dimensional space; attempt an extraction of a next task area from the third image; determine that the second observation area includes no task area; and select a third observation area in the three-dimensional space to acquire a fourth image, in response to determining that the second observation area includes no task area.

13 . The robot control system of claim 12 , wherein the third observation area is selected dynamically in the three-dimensional space, based on a current state of the observation area.

14 . The robot control system according to claim 12 , wherein the circuitry is configured to:
acquire extraction result history associated with one or more task area extractions attempted in previous observation areas; and calculate a score that indicates a possibility that the three-dimensional space includes the next task area to be extracted, based on the extraction result history, and wherein the third observation area is selected based on the score.

15 . The robot control system according to claim 12 , wherein the circuitry is configured to calculate a distribution of uncertainty that the three-dimensional space includes the next task area to be extracted, and
wherein the third observation area is selected based on the distribution of uncertainty.

16 . The robot control system according to claim 12 , wherein the circuitry is configured to:
control the image sensor to acquire images associated with a plurality of observation areas including the first observation area, the second observation area and the third observation area; and attempt an extraction of at least one task area at each of the plurality of observation areas.

17 . The robot control system according to claim 1 ,
wherein the task includes a process in which the robot is to contact the target object, and wherein the task area includes a region in the vicinity of the robot, in which the robot is operable to contact the target object to perform the task.

18 . The robot control system according to claim 1 ,
wherein the probability associated with the first image corresponds to a probability that the task area includes the target object, and wherein the probability is calculated based on pixels of the first image.

19 . A processor-executable method comprising:
acquiring a first image of an observation area in a vicinity of a robot, from an image sensor; calculating a probability that the observation area includes a task area in which the robot is to perform a task on a target object, based on the first image, wherein the calculating comprises comparing a configuration of an end effector of the robot with a structure of the target object included in the observation area based on the first image to calculate the probability; extracting the task area from the first image, based on the probability calculated; controlling the robot to cause the robot to approach the task area; acquiring a second image of the observation area from the image sensor, after the robot approaches the task area; calculating the probability that the observation area includes the task area, based on the sec

Assignees

• Yaskawa Electric Corp

Inventors

• KITTAKA Tatsuya

Classifications

• B25J13/089

  Determining the position of the robot with reference to its environment · CPC title

• B25J9/1679

  characterised by the tasks executed · CPC title

• B25J9/1664

  characterised by motion, path, trajectory planning · CPC title

• B25J19/023

  including video camera means · CPC title

• B25J9/1676

  Avoiding collision or forbidden zones · CPC title