Autonomous mobile robot control system, control method thereof, a non-transitory computer readable medium storing control program thereof, and autonomous mobile robot control device

Patent metadata

Field	Value
Publication number	US-11971721-B2
Application number	US-202117392541-A
Country	US
Kind code	B2
Filing date	Aug 3, 2021
Priority date	Aug 26, 2020
Publication date	Apr 30, 2024
Grant date	Apr 30, 2024

Abstract

To effectively enhance the operation efficiency of an autonomous mobile robot, an autonomous mobile robot control system includes a processor and a plurality of environmental cameras. The processor estimates a moving route of each of a plurality of moving bodies on the basis of characteristics of each of the plurality of moving bodies and sets a subset of the plurality of moving bodies whose moving routes overlap among the detected moving bodies as avoidance processing target moving bodies. The processor generates an avoidance procedure for the avoidance processing target moving bodies so the motion of the avoidance processing target moving bodies does not interfere with the motion of other avoidance target moving bodies.

First claim

What is claimed is:

1. An autonomous mobile robot control system comprising:
an autonomous mobile robot;
a processor configured to manage the autonomous mobile robot on the basis of a route plan defining a moving route of the autonomous mobile robot; and
a plurality of environmental cameras configured to capture images of a moving range of the autonomous mobile robot and transmit the captured images to the processor, wherein
the processor is configured to perform:
moving body detection processing that detects a plurality of moving bodies in the moving range on the basis of image information acquired using the plurality of environmental cameras,
moving body path estimation processing that estimates a moving route of each of the plurality of moving bodies on the basis of characteristics of each of the detected moving bodies,
avoidance procedure generation processing that sets a subset of the plurality of moving bodies whose moving routes overlap among the detected moving bodies as avoidance processing target moving bodies, and generates an avoidance procedure for the avoidance processing target moving bodies so as not to interfere with each other's motion, and
robot control processing that gives an instruction to perform avoidance behavior to the avoidance processing target moving bodies on the basis of the generated avoidance procedure, and wherein

the moving body path estimation processing further includes:
identifying the moving route of the autonomous mobile robot in a period later than the present time on the basis of route plan information indicating the route plan for the autonomous mobile robot, and
estimating a moving route of a first moving body of the plurality of moving bodies different from the autonomous mobile robot among the detected moving bodies on the basis of characteristics of the first moving body and an action history of the first moving body in a period earlier than the present time.

2. The autonomous mobile robot control system according to claim 1 , wherein the moving body detection processing detects, as the plurality of moving bodies, objects being movable, including at least one of a person, the autonomous mobile robot, and a carrying trolley from the image information.

3. The autonomous mobile robot control system according to claim 1 , wherein
the avoidance procedure generation processing performs moving body determination processing that determines whether at least one of the avoidance processing target moving bodies is a person or an object including a person, and
when at least one of the avoidance processing target moving bodies is the person or the object including a person, and another one of the avoidance processing target moving bodies is the autonomous mobile robot, the avoidance procedure generation processing generates the avoidance procedure for the autonomous mobile robot set as the avoidance processing target moving body so as not to interfere with motion of the avoidance processing target moving body whose moving route overlaps the moving route of the autonomous mobile robot.

4. The autonomous mobile robot control system according to claim 1 , wherein
the avoidance procedure generation processing performs moving body determination processing that determines whether at least one of the avoidance processing target moving bodies is a person or an object including a person,
when at least one of the avoidance processing target moving bodies is not the person or the object including a person, the avoidance procedure generation processing further performs priority identifying processing that identifies priorities of moving bodies included in the avoidance processing target moving bodies, and
the avoidance procedure generation processing generates the avoidance procedure for the autonomous mobile robot corresponding to the moving body with low priority so as not to interfere with motion of the other avoidance processing target moving bodies.

5. The autonomous mobile robot control system according to claim 1 , wherein the avoidance procedure generation processing further includes giving an instruction to issue a warning to a notification device installed in the moving range, and
the processor performs facility control processing that gives the instruction to issue the warning to the notification device on the basis of the avoidance procedure.

6. The autonomous mobile robot control system according to claim 1 , wherein
when there is a security area where access is limited to a specific person in the moving route which the autonomous mobile robot is to follow, the avoidance procedure generation processing performs person determination processing that determines whether the first moving body of the plurality of moving bodies is a person allowed to enter the security area,
when a person is an unauthorized person determined not to be allowed to enter the security area exists at a specified distance from the autonomous mobile robot, the avoidance procedure generation processing generates measures to prohibit entry into the security area as the avoidance procedure, and
the processor performs facility control processing that gives an instruction to take measures to ensure security of the security area on the basis of the avoidance procedure.

7. An autonomous mobile robot control method in an autonomous mobile robot control system including a processor configured to manage an autonomous mobile robot on the basis of a route plan defining a moving route of the autonomous mobile robot, and a plurality of environmental cameras configured to capture images of a moving range of the autonomous mobile robot and transmit the captured images to the processor, comprising:
detecting, by the processor, a plurality of moving bodies in the moving range on the basis of image information acquired using the plurality of environmental cameras;
estimating, by the processor, a moving route of each of the plurality of moving bodies on the basis of characteristics of each of the detected moving bodies;
setting, by the processor, a subset of the plurality of moving bodies whose moving routes overlap as avoidance processing target moving bodies, and generating an avoidance procedure for the avoidance processing target moving bodies so as not to interfere with each other's; and
giving, by the processor, an instruction to perform avoidance behavior to the avoidance processing target moving bodies on the basis of the generated avoidance procedure, wherein
the estimating, by the processor, of the moving route of each of the plurality of moving bodies further includes:
identifying, by processor, the moving route of the autonomous mobile robot in a period later than the present time on the basis of route plan information indicating the route plan for the autonomous mobile robot, and
estimating, by processor, a moving route of a first moving body of the plurality of moving bodies different from the autonomous mobile robot among the detected moving bodies on the basis of characteristics of the first moving body and an action history of the first moving body in a period earlier than the present time.

8. A non-transitory computer readable medium storing an autonomous mobile robot control program executed in a processor of an autonomous mobile robot control system including the processor configured to manage an autonomous mobile robot on the basis of a route plan defining a moving route of the autonomous mobile robot, and a plurality of environmental cameras configured to capture images of a moving range of the autonomous mobile robot and transmit the captured images to the processor, the autonomous mobile robot control program causing the processor to execute:
detecting a plurality of moving bodies in the moving range on the basis of image

Assignees

• Toyota Motor Co Ltd

Inventors

• Takai Tomohisa
• Yamaguchi Yuhei
• Toyoshima Satoshi
• Watanabe Yuta
• Taira Tetsuya
• Honda Mikio
• Oda Shiro
• Otsuki Nobuhisa

Classifications

• G05D1/0214 Primary

  in accordance with safety or protection criteria, e.g. avoiding hazardous areas (monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13) · CPC title

• B60W60/0011

  involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles · CPC title

• G05D1/0088

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title

• G05D1/0251

  extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision (stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593) · CPC title

• G05D1/0253 Primary

  extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow (determining position or orientation from images G06T7/70) · CPC title