Image processing system, image processing method, learned model generation method, and data set for learning

Patent metadata

Field	Value
Publication number	US-2022101552-A1
Application number	US-202017422029-A
Country	US
Kind code	A1
Filing date	Jan 24, 2020
Priority date	Feb 1, 2019
Publication date	Mar 31, 2022
Grant date	—

Abstract

In the image processing system according to the present invention, the data acquisition unit acquires a captured image showing a loading/unloading target of a transporting material of a work machine. The area-specifying unit specifies an area including the loading/unloading target from the captured image. The loading/unloading target-specifying unit specifies at least one predetermined surface of the loading/unloading target from the area including the loading/unloading target.

First claim

1 . An image processing system, comprising:
a data acquisition unit configured to acquire a captured image showing a dump body of a work machine; an area-specifying unit configured to specify an area including the dump body from the captured image; and a dump body surface-specifying unit configured to specify at least one predetermined surface of the dump body from the area specified by the area-specifying unit.

2 . The image processing system according to claim 1 ,
wherein the area-specifying unit specifies the area including the dump body based on a segmentation model, which is a learned model that outputs an output image in which a value of each of a plurality of pixels takes a value representing a type of an object shown in a pixel of the input image corresponding to the pixel by inputting an input image, and the captured image.

3 . The image processing system according to claim 1 , further comprising:
a posture-specifying unit configured to specify a position of the dump body based on the specified surface.

4 . The image processing system according to claim 3 ,
wherein the posture-specifying unit further specifies an azimuth direction and a posture of the dump body based on the specified surface.

5 . The image processing system according to claim 3 , further comprising:
a three-dimensional data generation unit configured to generate three-dimensional data representing a three-dimensional shape of a subject of the captured image, based on the captured image, wherein the posture-specifying unit specifies the position of the dump body based on the at least one predetermined surface in the three-dimensional data related to the area.

6 . The image processing system according to claim 5 ,
wherein the posture-specifying unit specifies the position of the dump body based on the at least one predetermined surface in the three-dimensional data related to the area and a target model which is a three-dimensional model indicating a shape of the dump body.

7 . The image processing system according to claim 3 ,
wherein the data acquisition unit acquires an image-capturing posture of an image-capturing device that captures the captured image, and the posture-specifying unit specifies a three-dimensional position of the dump body at a site based on the area and the image-capturing posture.

8 . The image processing system according to claim 5 ,
wherein the three-dimensional data generation unit generates transporting material three-dimensional data representing a three-dimensional position of a transporting material loaded in the dump body based on the captured image, the image processing system, further comprising: a distribution-specifying unit configured to generate distribution information indicating a distribution of an amount of the transporting material in the dump body based on the transporting material three-dimensional data in the dump body and a three-dimensional position of at least a part of the dump body.

9 . An image processing method, comprising the steps of:
acquiring a captured image showing a dump body for a transporting material of a work machine; specifying an area including the dump body from the captured image; and specifying at least one predetermined surface of the dump body from the area including the dump body.

10 . A method for generating a learned model of a segmentation model that outputs an area including the dump body by inputting a captured image showing a dump body for a transporting material of a work machine, the method comprising the steps of:
acquiring the captured image showing the dump body of the transporting material of the work machine; and generating the learned model by learning the segmentation model using the captured image showing the dump body and information indicating the area including the dump body shown in the captured image as a data set for learning.

11 . A data set for learning that is used in a computer and causes a segmentation model to learn, comprising:
a captured image showing a dump body for a transporting material of a work machine; and information indicating an area of the dump body of the transporting material of the work machine, wherein the data set for learning is used in processing of causing the segmentation model to learn by the computer.

12 . An image processing system, comprising:
a data acquisition unit configured to acquire a captured image showing a loading/unloading target of a transporting material of a work machine; an area-specifying unit configured to specify an area including the loading/unloading target from the captured image; and a loading/unloading target-specifying unit configured to specify at least one predetermined surface of the loading/unloading target from the area including the loading/unloading target.

Assignees

• Komatsu Mfg Co Ltd

Inventors

• Kawamoto Shun
• HAMADA Shintaro
• KAJIHARA Yosuke

Classifications

• G06T7/11 Primary

  Region-based segmentation · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06T17/20

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title