Image processing system, image processing method, learned model generation method, and data set for learning

Patent metadata

Field	Value
Publication number	US-12094151-B2
Application number	US-202017422029-A
Country	US
Kind code	B2
Filing date	Jan 24, 2020
Priority date	Feb 1, 2019
Publication date	Sep 17, 2024
Grant date	Sep 17, 2024

Abstract

In the image processing system according to the present invention, the data acquisition unit acquires a captured image showing a loading/unloading target of a transporting material of a work machine. The area-specifying unit specifies an area including the loading/unloading target from the captured image. The loading/unloading target-specifying unit specifies at least one predetermined surface of the loading/unloading target from the area including the loading/unloading target.

First claim

What is claimed is:

1. An image processing system, comprising:
a processor configured to:
acquire a captured image showing a dump body of a dump truck, the dump body being configured to accommodate a transporting material and being a loading/unloading target of the transporting material;
specify an area including the dump body from the captured image;
specify at least one predetermined surface of the dump body from the specified area;
generate three-dimensional data related to the area representing a three-dimensional shape of a subject of the captured image; and
specify a position of the dump body based on (i) the at least one predetermined surface in the three-dimensional data related to the area and (ii) a target model that is a three-dimensional model indicating a shape of the dump body.

2. The image processing system according to claim 1 , wherein the processor is configured to specify the area including the dump body based on a segmentation model that is a learned model configured to receive the captured image as an input image and to output an output image, and
wherein each of a plurality of pixels of the output image has a value representing a type of an object shown in a pixel of the input image corresponding to a pixel of the output image.

3. The image processing system according to claim 1 , wherein the processor is further configured to specify an azimuth direction and a posture of the dump body based on the at least one predetermined specified surface.

4. The image processing system according to claim 1 , wherein the processor is further configured to:
acquire an image-capturing posture of an image-capturing device that captures the captured image; and
specify a three-dimensional position of the dump body at a site based on the area and the image-capturing posture.

5. The image processing system according to claim 1 , wherein the processor is further configured to:
generate transporting material three-dimensional data representing a three-dimensional position of the transporting material loaded in the dump body based on the captured image; and
generate distribution information indicating a distribution of an amount of the transporting material in the dump body based on the transporting material three-dimensional data in the dump body and a three-dimensional position of at least a part of the dump body.

6. An image processing method, comprising:
acquiring a captured image showing a dump body of a dump truck, the dump body being configured to accommodate a transporting material and being a loading/unloading target of the transporting material;
specifying an area including the dump body from the captured image;
specifying at least one predetermined surface of the dump body from the specified area;
generating three-dimensional data related to the area representing a three-dimensional shape of a subject of the captured image; and
specifying a position of the dump body based on (i) the at least one predetermined surface in the three-dimensional data related to the area and (ii) a target model that is a three-dimensional model indicating a shape of the dump body.

7. The image processing method of claim 6 , wherein specifying the area including the dump body is performed based on a segmentation model that is a learned model configured to receive the captured image as an input image and to output an output image, and
wherein each of a plurality of pixels of the output image has a value representing a type of an object shown in a pixel of the input image corresponding to a pixel of the output image.

Assignees

• Komatsu Mfg Co Ltd

Inventors

• Kawamoto Shun
• HAMADA Shintaro
• KAJIHARA Yosuke

Classifications

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title

• G06N20/00

  Machine learning · CPC title