Photographing guide device

Patent metadata

Field	Value
Publication number	US-12223639-B2
Application number	US-201917417218-A
Country	US
Kind code	B2
Filing date	Dec 10, 2019
Priority date	Jan 10, 2019
Publication date	Feb 11, 2025
Grant date	Feb 11, 2025

Abstract

A photographing guide device includes a detection means and an output means. The detection means compares a candidate image of a structure captured from a capturing candidate position in a capturing candidate direction with a registered image of the structure captured from a given capturing position in a given capturing direction and stored in advance, thereby detecting a positional difference between the capturing candidate position and the given capturing position. The output means outputs information indicating the detected positional difference.

First claim

What is claimed is:

1. A photographing guide device comprising:
a memory storing program instructions; and
a processor coupled to the memory, wherein the processor is configured to execute the program instructions to:
extract a plurality of feature points from each of a candidate image of a structure captured from a capturing candidate position in a capturing candidate direction and a registered image of the structure captured from a given capturing position in a given capturing direction stored in advance;
associate each feature point extracted from the candidate image with a corresponding feature point extracted from the registered image, as a feature point pair, resulting in obtaining a plurality of feature point pairs;
when at least a predetermined number of the plurality of feature point pairs have been obtained, estimate a homography matrix of the candidate image and the registered image based on the plurality of feature point pairs;
detect a positional difference between the capturing candidate position and the given capturing position, and a direction difference between the capturing candidate direction and the given capturing direction, on a basis of the homography matrix; and
output information indicating the positional difference and the direction difference that have been detected, and indicating a synthetic image, wherein
detection of the positional difference includes transformation of the registered image on the basis of the homography matrix, and generation of the synthetic image in which the registered image after transformation and the candidate image are synthesized.

2. The photographing guide device according to claim 1 , wherein
detection of the positional difference includes generation of a crack image in which a crack on the structure is emphasized from the candidate image, and comparison of the generated crack image with a registered crack image that is generated from the registered image and stored in advance.

3. The photographing guide device according to claim 2 , wherein
detection of the positional difference includes estimation of a homography matrix of the crack image and the registered crack image, and detection of the positional difference on a basis of the homography matrix of the crack image and the registered crack image.

4. The photographing guide device according to claim 2 , wherein
detection of the positional difference includes estimation of a homography matrix of the crack image and the registered crack image, transformation of the registered crack image on a basis of the homography matrix of the crack image and the registered crack image, and generation of a synthetic image in which the registered crack image after transformation and the crack image are synthesized.

5. The photographing guide device according to claim 1 , wherein
detection of the positional difference includes comparison of a candidate three-dimensional shape obtained by measuring a three-dimensional shape of a peripheral environment from the capturing candidate position with a registered three-dimensional shape obtained by measuring the three-dimensional shape of the peripheral environment from the given capturing position and stored in advance.

6. The photographing guide device according to claim 5 , wherein
detection of the positional difference includes estimation of a geometric transformation matrix of the candidate three-dimensional shape and the registered three-dimensional shape, and detection of the positional difference on a basis of the geometric transformation matrix.

7. The photographing guide device according to claim 5 , wherein
detection of the positional difference includes estimation of a geometric transformation matrix of the candidate three-dimensional shape and the registered three-dimensional shape, transformation of the registered three-dimensional shape on a basis of the geometric transformation matrix, and generation of a synthetic image in which the registered three-dimensional shape after transformation and the candidate three-dimensional shape are synthesized.

8. A photographing guide method performed by a computer and comprising:
extracting a plurality of feature points from each of a candidate image of a structure captured from a capturing candidate position in a capturing candidate direction and a registered image of the structure captured from a given capturing position in a given capturing direction stored in advance;
associating each feature point extracted from the candidate image with a corresponding feature point extracted from the registered image, as a feature point pair, resulting in obtaining a plurality of feature point pairs;
when at least a predetermined number of the plurality of feature point pairs have been obtained, estimating a homography matrix of the candidate image and the registered image based on the plurality of feature point pairs;
detecting a positional difference between the capturing candidate position and the given capturing position, and a direction difference between the capturing candidate direction and the given capturing direction, on a basis of the homography matrix; and
outputting information indicating the positional difference and the direction difference that have been detected, and indicating a synthetic image, wherein
detection of the positional difference includes transformation of the registered image on the basis of the homography matrix, and generation of the synthetic image in which the registered image after transformation and the candidate image are synthesized.

9. A non-transitory computer-readable storage medium storing a program comprising instructions executable by a computer to perform processing comprising:
extracting a plurality of feature points from each of a candidate image of a structure captured from a capturing candidate position in a capturing candidate direction and a registered image of the structure captured from a given capturing position in a given capturing direction stored in advance;
associating each feature point extracted from the candidate image with a corresponding feature point extracted from the registered image, as a feature point pair, resulting in obtaining a plurality of feature point pairs;
when at least a predetermined number of the plurality of feature point pairs have been obtained, estimating a homography matrix of the candidate image and the registered image based on the plurality of feature point pairs;
detecting a positional difference between the capturing candidate position and the given capturing position, and a direction difference between the capturing candidate direction and the given capturing direction, on a basis of the homography matrix; and
outputting information indicating the positional difference and the direction difference that have been detected, and indicating a synthetic image, wherein
detection of the positional difference includes transformation of the registered image on the basis of the homography matrix, and generation of the synthetic image in which the registered image after transformation and the candidate image are synthesized.

Assignees

• Nec Corp

Inventors

• Takada Jun
• Nakano Gaku

Classifications

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06T2207/30184

  Infrastructure · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T7/30

  Determination of transform parameters for the alignment of images, i.e. image registration · CPC title

• G06T7/50

  Depth or shape recovery · CPC title