Moving body remote control system and moving body remote control method

Patent metadata

Field	Value
Publication number	US-2019011911-A1
Application number	US-201815921309-A
Country	US
Kind code	A1
Filing date	Mar 14, 2018
Priority date	Jul 7, 2017
Publication date	Jan 10, 2019
Grant date	—

Abstract

Provided is a moving body remote control system retains path information of a path along which a moving body autonomously travels and camera information of a camera provided in the moving body; stores the position of the moving body and an acquisition time thereof; outputs, upon receiving an image taken by the camera and an imaging time of the image, data of the received image, and stores the imaging time; estimates the position and direction of the moving body at the imaging time; identifies a part of the path included in the range of the image based on the path information, the camera information, and the estimated position and direction of the moving body; and converts the coordinates of the identified part of the path into coordinates of the image and outputs data of the image on which the path is superposed, at the position of the converted coordinates.

First claim

What is claimed is:

1 . A moving body remote control system having a processor, an interface unit that is coupled to the processor and that communicates with the moving body, a storage unit coupled to the processor, and a display unit coupled to the processor, wherein:
the storage unit retains path information showing the position of a path along which the moving body autonomously travels and camera information including the position, direction, and angle of view of a camera provided in the moving body; and the processor is configured: to store, upon receiving the position of the moving body and the acquisition time of the position via the interface unit, the received position and acquisition time in the storage unit; to output, upon receiving an image taken by the camera and an imaging time at which the image was taken via the interface unit, data of the received image to the display unit and store the imaging time in the storage unit; to estimate, based on the imaging time and the position of the moving body at the acquisition time, the position and direction of the moving body at the imaging time; to identify a part of the path included in the range of the image based on the path information, the camera information, and the estimated position and direction of the moving body; and to convert, based on the path information, the camera information, and the estimated position and direction of the moving body, the coordinates of the identified part of the path into coordinates of the image and output, to the display unit, data of the image on which the path is superposed, at the position of the converted coordinates.

2 . The moving body remote control system according to claim 1 , wherein:
the moving body remote control system further has a controller that is coupled to the processor and that receives an input of the operation of the moving body; and when the moving body determines that the moving body can no longer continue the autonomous travel on the path, the moving body transmits a remote control request to the moving body remote control system, the processor being configured: to estimate, upon receiving the remote control request from the moving body via the interface unit, the position and direction of the moving body at the imaging time, to identify a part of the path included in the range of the image, to output data of the image on which the path is superposed; and to transmit, after receiving the remote control request, a control signal depending on the operation inputted to the controller to the moving body via the interface unit.

3 . The moving body remote control system according to claim 1 , wherein
the processor converts, when the position of the moving body acquired at a time later than the imaging time is retained by the storage unit, the coordinates of the position of the moving body acquired at the time later than the imaging time into coordinates on the image and outputs, to the display unit, data of the image on which the moving body is superposed, at the position of the converted coordinates.

4 . The moving body remote control system according to claim 1 , wherein:
the storage unit retains information showing the width of the moving body; and the processor converts, based on the path information and the camera information, the width of the moving body into the corresponding width on the image to superpose a graphic shape having the converted width on the image as the path.

5 . The moving body remote control system according to claim 1 , wherein
the processor is configured: to estimate, based on the position of the moving body at the acquisition time, the direction of the moving body at the acquisition time; to estimate the position of the moving body at the imaging time by multiplying a difference between the position of the moving body acquired at a first time earlier than the imaging time and the position of the moving body acquired at a second time later than the imaging time with a ratio of the length from the first time to the photographing time of the image to the length from the first time to the second time to add the resultant value to the position of the moving body acquired at the first time; and to estimate the direction of the moving body at the imaging time by multiplying a difference between the direction of the moving body at the first time and the direction of the moving body at the second time with the ratio of the length from the first time to the imaging time to the length from the first time to the second time to add the resultant value to the direction of the moving body acquired at the first time.

6 . The moving body remote control system according to claim 1 , wherein
the processor is configured: to estimate the direction of the moving body at the acquisition time based on the position of the moving body at the acquisition time; and to estimate the position of the moving body acquired at a time closest to the imaging time and the direction of the moving body at the acquisition time as the direction of the moving body at the imaging time.

7 . The moving body remote control system according to claim 1 , wherein
the moving body remote control system further has a controller that is coupled to the processor and that receives an input of the control of the moving body, the processor being configured: to predict, when the controller receives an input of the operation to the moving body, a path of the moving body controlled in accordance with the inputted operation based on the estimated position and direction of the moving body and the inputted operation, to convert the predicted path to coordinates on the image, to output, to the display unit, data of the image on which the predicted path is superposed, at the position of the converted coordinates on the image, and to transmit a control signal depending on the inputted operation to the moving body via the interface unit.

8 . A moving body remote control method using a moving body remote control system having a processor, an interface unit that is coupled to the processor and that communicates with a moving body, a storage unit coupled to the processor, and a display unit coupled to the processor, wherein
the storage unit retains path information showing the position of a path along which the moving body autonomously travels and camera information including the position, direction, and angle of view of a camera provided in the moving body, the moving body remote control method including: a step of storing, by the processor, upon receiving the position of the moving body and the acquisition time of the position via the interface unit, the received position and acquisition time in the storage unit; a step of outputting, by the processor, upon receiving an image taken by the camera and an imaging time at which the image was taken via the interface unit, data of the received image to the display unit and store the imaging time in the storage unit; a step of estimating, by the processor, the position and direction of the moving body at the imaging time based on the imaging time and the position of the moving body at the acquisition time; a step of identifying, by the processor, a part of the path included in the range of the image based on the path information, the camera information, and the estimated position and direction of the moving body; and a step of converting, by the processor, the coordinates of the identified part of the path into coordinates on the image based on the path information, the camera information, and the estimated position and direction of the moving body to output, to the display unit, data of the image on which the path is superposed, at the position of the converted coordinates.

Assignees

• Hitachi Ltd

Inventors

• Kubo Hiroyuki
• Endo Hideki
• Yunoki Shoji

Classifications

• B60W50/00

  Details of control systems for road vehicle drive control not related to the control of a particular sub-unit {, e.g. process diagnostic or vehicle driver interfaces} · CPC title

• B60W2050/0064

  using a remote, e.g. cordless, transmitter or receiver unit, e.g. remote keypad or mobile phone · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G08G1/09623

  Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle (G08G1/0967 takes precedence) · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title