Remote operation assistance server and remote operation asistance system

Patent metadata

Field	Value
Publication number	US-2024141616-A1
Application number	US-202118278209-A
Country	US
Kind code	A1
Filing date	Dec 13, 2021
Priority date	Mar 19, 2021
Publication date	May 2, 2024
Grant date	—

Abstract

Provided is a system or the like capable of reducing mental burdens on an operator who remotely operates a work machine, when an image range acquired by a real machine image pickup device loaded on the work machine is changed. A previous notice indicating a future change mode of a real space area projected in a work environment image displayed at a remote image output device 221 is output to a remote output interface 220. It is possible to make the operator who encounters a virtual sound source Vss displaced according to a displacement mode of the real space area and/or an icon image A1 in an arrow shape indicating the displacement mode of the real space area recognize beforehand that the real space area projected in the work environment image is to be displaced according to the displacement mode in the future.

First claim

1 . A remote operation assistance server, which is a server for assisting a remote operation of a work machine using a remote operation device, comprising:
a first assistance processing element configured to acquire a picked-up image indicating an environment of the work machine through a real machine image pickup device loaded on the work machine, and make a work environment image according to the picked-up image be displayed at a remote image output device configuring the remote operation device; and a second assistance processing element configured to recognize a future change mode of a real space area projected in the work environment image displayed at the remote image output device by the first assistance processing element, and make a remote output interface configuring the remote operation device output a previous notice indicating the future change mode of the real space area.

2 . The remote operation assistance server according to claim 1 ,
wherein the second assistance processing element recognizes the future change mode of the real space area projected in the work environment image, according to a specified state detected by a real machine periphery monitoring device loaded on the work machine.

3 . The remote operation assistance server according to claim 1 ,
wherein the second assistance processing element recognizes a future displacement mode of the real space area as the future change mode of the real space area, and makes an output part of the previous notice configured by the remote output interface be displaced according to the future displacement mode of the real space area.

4 . The remote operation assistance server according to claim 3 ,
wherein the second assistance processing element further recognizes a future scaling mode of the real space area as the future change mode of the real space area, and makes the remote output interface displace the output part of the previous notice according to the future scaling mode of the real space area before or after making the output part of the previous notice configured by the remote output interface be displaced according to the future displacement mode of the real space area.

5 . The remote operation assistance server according to claim 3 ,
wherein the second assistance processing element makes a remote sound output device configuring the remote output interface displace a virtual sound source that emits sound as the previous notice.

6 . The remote operation assistance server according to claim 3 ,
wherein the second assistance processing element recognizes the future displacement mode of the real space area as the future change mode of the real space area according to a switching mode from one real machine image pickup device of the plurality of real machine image pickup devices loaded on the work machine to another real machine image pickup device.

7 . The remote operation assistance server according to claim 6 ,
wherein the second assistance processing element recognizes a future scaling mode of the real space area as the future change mode of the real space area, and makes the remote output interface change an output mode of the previous notice according to the future scaling mode of the real space area.

8 . The remote operation assistance server according to claim 7 ,
wherein the second assistance processing element makes a remote sound output device configuring the remote output interface change at least one of a height of a sound pressure and a height of a frequency of sound as the previous notice.

9 . A remote operation assistance system configured by the remote operation assistance server according to claim 8 and at least one of the remote operation device and the work machine.

Assignees

• Kobelco Constr Mach Co Ltd

Inventors

• Morita Yuki
• Otani Masaki
• Sawada Yusuke
• Kamimura Yusuke
• Yamazaki Yoichiro

Classifications

• E02F9/205 Primary

  Remotely operated machines, e.g. unmanned vehicles (E02F3/8866 takes precedence) · CPC title

• E02F9/261

  Surveying the work-site to be treated · CPC title

• E02F3/435

  for dipper-arms, backhoes or the like · CPC title