Image capturing system for shape measurement of structure, method of capturing image of structure for shape measurement of structure, on-board control device, remote control device, program, and storage medium

Patent metadata

Field	Value
Publication number	US-10218893-B2
Application number	US-201615760276-A
Country	US
Kind code	B2
Filing date	Oct 11, 2016
Priority date	Oct 27, 2015
Publication date	Feb 26, 2019
Grant date	Feb 26, 2019

Abstract

An image capturing system for shape measurement includes: an image capturing device configured to capture an image of a structure; an air vehicle; an on-board control device configured to control the image capturing device and the air vehicle in accordance with an image capturing scenario; and a remote control device configured to create an image capturing scenario and transfer the created image capturing scenario to the on-board control device. The air vehicle autonomously flies to image capturing points sequentially so as to capture images. The captured data is transmitted to the remote control device via a wireless LAN.

First claim

The invention claimed is:

1. An image capturing system for shape measurement of a structure, the image capturing system comprising:
an image capturing device configured to capture an image of a target structure to measure a shape of the target structure;
an air vehicle having the image capturing device mounted thereon, the air vehicle being configured to fly and be unmoved in air;
a distance measurement device mounted on the air vehicle and configured to measure a distance between the air vehicle and the target structure;
an image capturing scenario storage configured to store an image capturing scenario, the image capturing scenario including:
a plurality of image capturing points at each of which the air vehicle is unmoved in air with a distance from the target structure being maintained when capturing the image of the target structure to measure the shape of the target structure with the target structure being unmoved, and
a flight route set in accordance with a positional relation between the target structure and each of the image capturing points or coordinates of each of the image capturing points such that the air vehicle having the image capturing device mounted thereon and configured to capture the image of the target structure flies via the image capturing points sequentially;

an on-board control device mounted on the air vehicle, the on-board control device including:
an image capturing controller configured to control the image capturing device in accordance with the image capturing scenario, and
a flight controller configured to control the air vehicle in accordance with the image capturing scenario based on the distance measured by the distance measurement device; and

a remote control device including:
a scenario creator configured to create the image capturing scenario based on the image capturing points, and
a scenario transferor configured to transfer the image capturing scenario created by the scenario creator to the on-board control device to store the image capturing scenario in the image capturing scenario storage,

the scenario creator configured to:
check whether or not a path connecting a first image capturing point to a second image capturing point in a straight line meets the target structure, the first image capturing point and the second image capturing point being different image capturing points,
when the path does not meet the structure, create the flight route including the path connecting the first image capturing point to the second image capturing point in the straight line, and
when the path meets the structure, create the flight route including a path avoiding the target structure in flight between the first image capturing point and the second image capturing point.

2. The image capturing system for the, shape measurement of the structure according to claim 1 , wherein
the remote control device further includes a simulator configured to perform a simulation of the image capturing scenario created by the scenario creator and present a result of the simulation to an operator, and
the simulator is configured to present to the operator at least one of presence/absence of an obstacle on the flight route, a flight time estimate value, and a battery remaining amount estimate value after the flight of the air vehicle on the flight route.

3. The image capturing system for the shape measurement of the structure according to claim 2 , further comprising an obstacle detector mounted on the air vehicle and configured to detect an obstacle around the air vehicle, wherein
when the obstacle detector detects an obstacle on the flight route during flight, the flight controller is configured to change the flight route to avoid the obstacle to reach the image capturing point.

4. The image capturing system for the shape measurement of the structure according to claim 1 , further comprising:
a plurality of the image capturing devices;
a plurality of the air vehicles having the respective image capturing devices mounted thereon;
a plurality of the on-board control devices mounted on the respective air vehicles; and
the remote control device including:
the scenario creator configured to create divided scenarios based on the plurality of image capturing points, the divided scenarios being respective image capturing scenarios for the air vehicles, and
the scenario transferer configured to transfer the divided scenarios to the on-board control devices of the corresponding air vehicles; wherein

each of the plurality of image capturing points is included in the image capturing scenario of any one of the divided scenarios.

5. The image capturing system for the shape measurement of the structure according to claim 4 , wherein the plurality of image capturing points include simultaneous image capturing points at which images are determined to be captured simultaneously, and the respective simultaneous image capturing points are included in different divided scenarios.

6. The image capturing system for the shape measurement of the structure according to claim 1 , further comprising an obstacle detector mounted on the air vehicle and configured to detect an obstacle around the air vehicle, wherein
when the obstacle detector detects an obstacle on the flight route during flight, the flight controller is configured to change the flight route to avoid the obstacle to reach the image capturing point.

7. A remote control device comprising:
a scenario creator configured to create an image capturing scenario, the image capturing scenario including:
a plurality of image capturing points at each of which an air vehicle is unmoved in air with a distance from a target structure being Maintained When capturing an image of the target structure to measure the shape of the target structure with the target structure being unmoved, the air vehicle having an image capturing device mounted thereon to measure the shape of the target structure, the image capturing device configured to capture the image of the target structure, the air vehicle being configured to fly and be unmoved in air, the air vehicle further having a distance measurement device amounted thereon and configured to measure a distance between the air vehicle and the target structure, the distance from the target structure being maintained based on the measured distance, and
a flight route set in accordance with a positional relation between the target structure and each of the image capturing points or coordinates of each of the image capturing points such that the air vehicle having the image capturing device mounted thereon and configured to capture the image of the target structure flies via the image capturing points sequentially; and

a scenario transferer configured to transfer the image capturing scenario created by the scenario creator to the air vehicle,
the scenario creator being configured to:
check whether or not a path connecting a first image capturing point to a second image capturing point in a straight line meets the target structure, the first image capturing point and the second image capturing point being different image capturing points,
when the path does not meet the structure, create the flight route including the path connecting the first image capturing point to the second image capturing point in the straight line, and
when the path meets the target structure, create the flight route including a path avoiding the target structure in flight between the first image capturing point and the second image capturing point.

8. An on-board control device comprising:
an image capturing scenario storage configured to store the image capturing scenario transferred from the scenario transferer of the remote control device recited in claim 7 ;
an

Assignees

• Mitsubishi Electric Corp

Inventors

• Iwakura Takashi
• Ishida Takayuki

Classifications

• B64U70/83

  using parachutes, balloons or the like · CPC title

• H04N23/57

  Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices · CPC title

• H04N23/617

  Upgrading or updating of programs or applications for camera control · CPC title

• H04N23/60

  Control of cameras or camera modules · CPC title

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title