Calibration device and calibration method

Patent metadata

Field	Value
Publication number	US-11900669-B2
Application number	US-202117528582-A
Country	US
Kind code	B2
Filing date	Nov 17, 2021
Priority date	Dec 4, 2020
Publication date	Feb 13, 2024
Grant date	Feb 13, 2024

Abstract

A calibration device performs a calibration between a plurality of imaging devices, each of which outputs field-of-view information which is information on a field-of-view of the imaging device itself. The field-of-view information contains a bitmap image and a range image. The calibration device includes: a state estimation part configured to detect, in a field of view of a first imaging device, an image of a second imaging device, and estimate a relative position and a relative attitude of the second imaging device with respect to the first imaging device, based on the detected image; and a transformation information calculation part configured to calculate transformation information between a coordinate system of the first imaging device and a coordinate system of the second imaging device, based on the estimated relative position and relative attitude.

First claim

The invention claimed is:

1. A calibration device which performs a calibration between a plurality of imaging devices, each of which outputs field-of-view information which is information on a field-of-view of the imaging device itself, the field-of-view information including a bitmap image and a range image, the calibration device comprising:
a state estimation part configured to
detect, in a field of view of a first imaging device, a single image of a second imaging device, and
estimate a relative position and a relative attitude of the second imaging device with respect to the first imaging device, solely based on the single detected image;
a transformation information calculation part configured to calculate transformation information between a coordinate system of the first imaging device and a coordinate system of the second imaging device, based on the estimated relative position and relative attitude, wherein the range image is an image in which range information is added to each pixel of the range image; and

wherein each of the imaging devices is an aerial imaging device without a stationary position; and
wherein each of the imaging devices is capable of measuring a distance between itself and a target and distance between the imaging devices is not known beforehand.

2. The calibration device according to claim 1 , further comprising
a field-of-view integrate processing part configured to make a point contained in field-of-view information outputted from the first imaging device associated with a point contained in field-of-view information outputted from the second imaging device, using the calculated transformation information, and thereby integrate the field-of-view information outputted from the first imaging device with the field-of-view information outputted from the second imaging device.

3. The calibration device according to claim 2 ,
wherein the field-of-view integrate processing part uses the transformation information as an initial value of an iterative computation processing in which the two points in each of the field-of-view information are made to be associated with each other.

4. The calibration device according to claim 1 ,
wherein the state estimation part is configured to: extract a centroid and a major component from a spatial distribution of range information of the second imaging device contained in the range image outputted by the first imaging device; and estimate a relative position and a relative attitude of the second imaging device.

5. The calibration device according to claim 1 ,
wherein the state estimation part is configured to estimate a relative attitude of the second imaging device, using a machine learning model which learns from training data in which an attitude of the second imaging device is a correct answer data from among images of the second imaging device contained in bitmap images outputted by the first imaging device.

6. The calibration device according to claim 1 ,
wherein the transformation information is a translation vector from the first imaging device to the second imaging device and a rotation matrix by which an orientation of the first imaging device is aligned with an orientation of the second imaging device.

7. The calibration device according to claim 1 ,
further comprising a correction processing part,
wherein the state estimation part is configured to: detect an image of the first imaging device in a field of view of the second imaging device; and estimate a relative position and a relative attitude of the first imaging device with respect to the second imaging device,
wherein the transformation information calculation part is configured to calculate second transformation information based on the relative position and the relative attitude of the first imaging device, and
wherein the correction processing part is configured to correct at least one of the transformation information and the second transformation information, based on a difference calculated from the transformation information and the second transformation information.

8. The calibration device according to claim 7 ,
wherein the difference is a difference calculated from at least one of:
(1) a difference: between a relative position of the second imaging device with reference to the first imaging device, the relative position being estimated from a field-of-view image outputted by the first imaging device; and a relative position of the second imaging device with respect to the first imaging device calculated from the second transformation information, and
(2) a difference between: a relative position of the first imaging device with respect to the second imaging device, the relative position being estimated from a field-of-view image outputted by the second imaging device; and a relative position of the first imaging device with respect to the second imaging device, the relative position being calculated from the transformation information.

9. The calibration device according to claim 7 ,
wherein the correction processing part is configured to repeat the correction until the difference takes a value equal to or smaller than a prescribed value.

10. A calibration method of a calibration device, comprising the steps of:
detecting, in a field of view of a first imaging device, an image of a second imaging device, the first imaging device being an imaging device outputting field-of-view information including a bitmap image and a range image, and estimating a relative position and a relative attitude of the second imaging device with respect to the first imaging device solely based on a single detected image of the second device; and
calculating transformation information between a coordinate system of the first imaging device and a coordinate system of the second imaging device, based on the estimated relative position and relative attitude,

wherein the range image is an image in which range information is added to each pixel of the range image, and
wherein each of the imaging devices is an aerial imaging device without a stationary position, and
wherein each of the imaging devices is capable of measuring a distance between itself and a target and distance between the imaging devices is not known beforehand.

Assignees

• Hitachi Ltd

Inventors

• Mitani Keiichi
• Tajima Kazuyuki
• Yamazaki Kazuyoshi

Classifications

• G06V20/13 Primary

  Satellite images · CPC title

• G06F18/00

  Pattern recognition · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F18/231

  Hierarchical techniques, i.e. dividing or merging pattern sets so as to obtain a dendrogram · CPC title

• G06V10/147

  Details of sensors, e.g. sensor lenses (fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19) · CPC title