Image generation apparatus and paver operation assistance system

Patent metadata

Field	Value
Publication number	US-9734413-B2
Application number	US-201514667837-A
Country	US
Kind code	B2
Filing date	Mar 25, 2015
Priority date	Nov 8, 2012
Publication date	Aug 15, 2017
Grant date	Aug 15, 2017

Abstract

An image generation apparatus includes a processor that generates an output image based on input images captured by multiple imaging units mounted on a paver including a hopper, a tractor, and a screed. The output image includes a hopper image representing a downward view of the hopper, a left-side surrounding image representing a downward view of a surrounding area on the left side in a movement direction of the paver, a right-side surrounding image representing a downward view of a surrounding area on the right side in the moving direction of the paver, and an illustration image of the tractor. The processor arranges the hopper image, the left-side surrounding image, the right-side surrounding image, and the illustration image such that the output image represents a downward view of the paver.

First claim

What is claimed is:

1. An image generation apparatus, comprising:
a processor that generates an output image based on input images captured by multiple imaging units mounted on a paver including a hopper, a tractor, and a screed,
wherein the output image includes a hopper image representing a downward view of the hopper, a left-side surrounding image representing a downward view of a surrounding area on a left side in a movement direction of the paver, a right-side surrounding image representing a downward view of a surrounding area on a right side in the movement direction of the paver, and an illustration image of the tractor; and
wherein the processor arranges the hopper image, the left-side surrounding image, the right-side surrounding image, and the illustration image such that the hopper image is disposed on an upper side of the illustration image, the left-side surrounding image is disposed to a left of the illustration image, the right-side surrounding image is disposed to a right of the illustration image, and the output image represents a downward view of the paver.

2. The image generation apparatus as claimed in claim 1 , wherein the processor
places the hopper image at a top center of the output image;
places the left-side surrounding image, which includes an image of a left end of the screed and an image of a left end of the hopper that is fully opened, in a left side of the output image; and
places the right-side surrounding image, which includes an image of a right end of the screed and an image of a right end of the hopper that is fully opened, in a right side of the output image.

3. The image generation apparatus as claimed in claim 1 , wherein the processor generates each of the hopper image, the left-side surrounding image, and the right-side surrounding image based on one of the input images that is captured by a corresponding one of the imaging units.

4. The image generation apparatus as claimed in claim 1 , wherein the processor performs distortion correction on at least one of the hopper image, the left-side surrounding image, and the right-side surrounding image.

5. The image generation apparatus as claimed in claim 1 , further comprising:
a storage unit that stores an input-output image correspondence map indicating a correspondence between coordinates of the input images and coordinates of the output image,
wherein the processor generates the output image by associating the coordinates of the input images with the coordinates of the output image according to the input-output image correspondence map.

6. The image generation apparatus as claimed in claim 1 , further comprising:
a front camera mounted on a front side of the tractor, a left-side camera mounted on a left side of the tractor, and a right-side camera mounted on a right side of the tractor that comprise the imaging units,
wherein the processor generates the hopper image based on an input image captured by the front camera, generates the left-side surrounding image based on an input image captured by the left-side camera, and generates the right-side surrounding image based on an input image captured by the right-side camera.

7. The image generation apparatus as claimed in claim 6 , wherein an imaging range of the front camera covers an entirety of the hopper.

8. The image generation apparatus as claimed in claim 6 , wherein
the processor generates the output image such that a non-rectangular area of the input image captured by the left-side camera corresponds to the left-side surrounding image with a rectangular shape and a non-rectangular area of the input image captured by the right-side camera corresponds to the right-side surrounding image with a rectangular shape.

9. The image generation apparatus as claimed in claim 1 ,
wherein the processor
places the hopper image at a top center of the output image;
places the left-side surrounding image in a position that is in a left side of the output image and to a left of the hopper image, and
places the right-side surrounding image in a position that is in a right side of the output image and to a right of the hopper image;

wherein the hopper image and the left-side surrounding image are two images that respectively include first image parts corresponding to a same first area in a three-dimensional space such that the first image parts are displayed separately; and
wherein the hopper image and the right-side surrounding image are two images that respectively include second image parts corresponding to a same second area in the three-dimensional space such that the second image parts are displayed separately.

10. A paver operation assistance system, comprising:
an image generation apparatus including
a processor that generates an output image based on input images captured by multiple imaging units mounted on a paver including a hopper, a tractor, and a screed,
the output image including a hopper image representing a downward view of the hopper, a left-side surrounding image representing a downward view of a surrounding area on a left side in a movement direction of the paver, a right-side surrounding image representing a downward view of a surrounding area on a right side in the movement direction of the paver, and an illustration image of the tractor; and

a display unit that is disposed near an operation unit for moving or operating the paver, and displays the output image generated by the processor of the image generation apparatus,
wherein the processor arranges the hopper image, the left-side surrounding image, the right-side surrounding image, and the illustration image such that the hopper image is disposed on an upper side of the illustration image, the left-side surrounding image is disposed to a left of the illustration image, the right-side surrounding image is disposed to a right of the illustration image, and the output image represents a downward view of the paver.

Assignees

• Sumitomo Heavy Industries

Inventors

• Ichikawa Tomoko

Classifications

• G06K9/00791 Primary

  Physics · mapped topic

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• E01C23/163

  for marking-out the location of the markings; Guiding or steering devices for marking apparatus, e.g. sights · CPC title

• E01C19/48 Primary

  for laying-down the materials and consolidating them, or finishing the surface {, e.g. slip forms therefor, forming kerbs or gutters in a continuous operation in situ (E01C19/002, E01C23/065 take precedence; devices for guiding or controlling the machines along a predetermined path E01C19/004)} · CPC title

• E01C19/004

  Devices for guiding or controlling the machines along a predetermined path (for graders or bulldozers E02F3/841; for other soil-shifting machines E02F9/2045) · CPC title