Work assistance system

Patent metadata

Field	Value
Publication number	US-2025013418-A1
Application number	US-202218697929-A
Country	US
Kind code	A1
Filing date	Oct 5, 2022
Priority date	Oct 5, 2021
Publication date	Jan 9, 2025
Grant date	—

Abstract

A work assistance system includes a first system used by a first user, and a second system used by a second user who gives an instruction about work in a space where the first user is present. The second system includes: a second display unit that displays an image in the space captured by the first system; a data generation unit that generates instruction reference position data indicating an instruction reference position corresponding to image capturing position data and instruction position data indicating an instruction position based on an operation of designating the instruction position on the image; and a second communication unit that outputs the instruction reference position data and the instruction position data. The virtual object generation unit generates a work assistance virtual object based on the instruction reference position data and the instruction position data.

First claim

1 . A work assistance system comprising:
a first system used by a first user; and a second system used by a second user who gives an instruction about work in a space where the first user is present, wherein the first system includes: an image capturing unit that captures an image in the space; a virtual object generation unit that generates a work assistance virtual object that assists the work in the space; a first display unit that displays the work assistance virtual object superimposed on the space; and a first communication unit that outputs image capturing position data indicating an image capturing position of the image capturing unit in the space and the image, the second system includes: a second display unit that displays the image; a data generation unit that generates instruction reference position data indicating an instruction reference position corresponding to the image capturing position data and instruction position data indicating an instruction position based on an operation of designating the instruction position on the image; and a second communication unit that outputs the instruction reference position data and the instruction position data, and the virtual object generation unit generates the work assistance virtual object based on the instruction reference position data and the instruction position data.

2 . The work assistance system according to claim 1 , wherein
when each of the operations of designating the instruction position on each of a plurality of the images with the image capturing position data different from each other is an operation of designating the same instruction position, the data generation unit generates a plurality of combined data of the instruction reference position data and the instruction position data in response to each of the operations, and the virtual object generation unit calculates a first intersection point of straight lines passing through positions indicated by the instruction reference position data and the instruction position data of each of the plurality of combined data, and generates the work assistance virtual object based on the first intersection point.

3 . The work assistance system according to claim 2 , wherein the virtual object generation unit generates, as the work assistance virtual object, a virtual object along a line segment extending from the instruction reference position indicated by the instruction reference position data of each of the plurality of combined data to the first intersection point.

4 . The work assistance system according to claim 2 , wherein the virtual object generation unit generates a virtual mark placed at the first intersection point as the work assistance virtual object.

5 . The work assistance system according to claim 1 , wherein
the first system further includes a scan unit that generates three-dimensional scan data about a real object in the space by scanning in the space, and the virtual object generation unit calculates a second intersection point of an outer surface of the real object based on the three-dimensional scan data and a straight line passing through positions indicated by the instruction reference position data and the instruction position data, and generates the work assistance virtual object based on the second intersection point.

6 . The work assistance system according to claim 5 , wherein the virtual object generation unit generates, as the work assistance virtual object, a virtual object along a line segment extending from the instruction reference position indicated by the instruction reference position data to the second intersection point.

7 . The work assistance system according to claim 5 , wherein the virtual object generation unit generates a virtual mark placed at the second intersection point as the work assistance virtual object.

8 . The work assistance system according to claim 1 , wherein
when the operation of designating the instruction position is continuously performed on the image, the data generation unit generates a plurality of the continuous instruction position data in response to the operation, and the virtual object generation unit generates, as the work assistance virtual object, a plurality of virtual objects along a plurality of straight lines extending from the instruction reference position indicated by the instruction reference position data toward the instruction position indicated by each of the plurality of instruction position data.

9 . The work assistance system according to claim 1 , further comprising a server device connected to the first communication unit and the second communication unit via a communication network to allow data communication,
wherein the server device includes a server storage unit that stores the image capturing position data, the image, the instruction reference position data, and the instruction position data.

Assignees

• Kawasaki Heavy Ind Ltd

Inventors

• Nakagawa Atsuki
• Nakamura Naohiro
• Iwamoto Masahiro
• Shikoda Shigekazu
• Tani Osamu
• Akamatsu Masahiko
• Yonemoto Singo

Classifications

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06F3/1454 Primary

  involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay · CPC title

• G06T19/00

  Manipulating three-dimensional [3D] models or images for computer graphics · CPC title

• G06F3/005

  Input arrangements through a video camera · CPC title

• G06F3/01

  Input arrangements or combined input and output arrangements for interaction between user and computer (G06F3/16 takes precedence) · CPC title