Remote operation system of forklift

Patent metadata

Field	Value
Publication number	US-12264053-B2
Application number	US-202217878918-A
Country	US
Kind code	B2
Filing date	Aug 2, 2022
Priority date	Jun 3, 2022
Publication date	Apr 1, 2025
Grant date	Apr 1, 2025

Abstract

A remote operation system 1 includes: an image capturing device 20 , provided at a forklift F; a head-mounted type image display device 10 , mounted to a head of an operator O; a command reception device 11 , receiving a remote operation command by the operator O; and a synthesized remote image generation device 12 A, obtaining a content of the remote operation command received by the command reception device 11 , an action state of the forklift F acting according to the remote operation command, and a remote image captured by the image capturing device 20 , and generating a synthesized remote image by synthesizing a command display image indicating the content of the remote operation command and an action display image indicating the action state of the forklift F into the remote image. The head-mounted type image display device 10 displays the generated synthesized remote image before eyes of the operator O.

First claim

What is claimed is:

1. A remote operation system, comprising:
an image capturing device, provided at a forklift at a place separate from an operator;
a head-mounted image display device, mounted to a head of the operator;
a controller comprising at least one button, within reach of a hand of the operator and receiving a remote operation command by the operator; and
a processor, configured to:
generate a command display image indicating a content of the remote operation command received by the controller,
generate an action display image indicating an action state of the forklift acting in accordance with the remote operation command, obtain a remote image captured by the image capturing device, and
generate a synthesized remote image by synthesizing the command display image and the action display image into the remote image,
wherein the head-mounted image display device displays the synthesized remote image that is generated by the processor before eyes of the operator,
wherein the processor is further configured to detect a posture of the head of the operator, the processor obtains the posture, extracts a portion of the remote image in accordance with the posture, and generates the synthesized remote image by synthesizing the command display image and the action display image into the portion of the remote image that is extracted from the remote image in accordance with the posture, wherein a size of the portion of the remote image is unchanged in response to change of the posture,
wherein the synthesized remote image comprises the content of the remote operation command, the action state of the forklift and the remote image.

2. The remote operation system as claimed in claim 1 , wherein the processor is disposed on a side of the operator, and obtains the action state of the forklift and the remote image from the forklift via an Internet line.

3. The remote operation system as claimed in claim 1 , wherein the processor is disposed at the forklift, and obtains the content of the remote operation command from the controller via an Internet line.

4. The remote operation system as claimed in claim 1 , wherein the processor is disposed at a place separate from the operator and the forklift, obtains the action state of the forklift and the remote image from the forklift via an Internet line, and obtains the content of the remote operation command from the controller via the Internet line.

5. The remote operation system as claimed in claim 1 , wherein the controller comprises four buttons arranged in a cross shape and corresponding to lifting and lowering of a fork provided at the forklift and forward and backward leaning of a mast provided at the forklift,
wherein the command display image comprises four cargo symbols arranged in a cross shape and corresponding to the four buttons, and
when any one of the four buttons is operated, an appearance of the cargo symbol corresponding to the button changes.

6. The remote operation system as claimed in claim 1 , wherein the controller comprises an analog stick for providing a command on a traveling direction and a traveling speed of the forklift, and
the command display image comprises a vehicle symbol with the forklift set as a motif and a traveling symbol whose relative position from the vehicle symbol changes in response to a tilting direction and a tilting amount of the analog stick.

Assignees

• Mitsubishi Logisnext Co Ltd

Inventors

• Kiwata Keisuke

Classifications

• E02F9/267

  Diagnosing or detecting failure of vehicles · CPC title

• E02F9/261

  Surveying the work-site to be treated · CPC title

• E02F9/205

  Remotely operated machines, e.g. unmanned vehicles (E02F3/8866 takes precedence) · CPC title

• B66F9/24

  Electrical devices or systems · CPC title

• G05D1/0038

  by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation (images analyzed by a computer and used for automatic navigation G05D1/0246) · CPC title