Automatic multi-image 3D ground control point extraction

Patent metadata

Field	Value
Publication number	US-11538135-B2
Application number	US-202016800846-A
Country	US
Kind code	B2
Filing date	Feb 25, 2020
Priority date	Feb 25, 2020
Publication date	Dec 27, 2022
Grant date	Dec 27, 2022

Abstract

Discussed herein are devices, systems, and methods for multi-image ground control point (GCP) determination. A method can include extracting, from a first image including image data of a first geographical region, a first image template, the first image template including a contiguous subset of pixels from the first image and a first pixel of the first image indicated by the GCP, predicting a first pixel location of the GCP in a second image, the second image including image data of a second geographical overlapping with the first geographical region, extracting, from the second image, a second image template, the second image template including a contiguous subset of pixels from the second image and a second pixel corresponding to the pixel location, identifying a second pixel of the second image corresponding to a highest correlation score, and adding a second pixel location of the identified pixel to the GCP.

First claim

What is claimed is:

1. A computer-implemented method for multi-image three-dimensional (3D) ground control point (GCP) determination, the method comprising:
extracting, from a first two-dimensional (2D) image including image data of a first geographical region, a first image template, the first image template including a contiguous subset of pixels from the first 2D image and a first pixel of the contiguous subset of pixels of the first 2D image indicated by the GCP, wherein the GCP is a five-tuple that associates a point in a three-dimensional (3D) point set with the first pixel of the first image;
predicting a first pixel location of the first pixel of the GCP in a second 2D image, the second 2D image including image data of a second geographical region overlapping with the first geographical region;
extracting, from the second 2D image, a second image template, the second image template including a contiguous subset of pixels from the second 2D image and a second pixel of the contiguous subset of pixels of the second 2D image corresponding to the first pixel location;
determining respective correlation scores, using a normalized cross-correlation, for the first image template centered at a variety of pixels of the contiguous subset of pixels of the second image template;
comparing a highest correlation score of the correlation scores to a first threshold value;
discarding a location of the second pixel responsive to determining a ratio is less than the first threshold value;
identifying an undiscarded highest correlation pixel of the second 2D image corresponding to a remaining highest correlation score; and
adding a pixel location of the undiscarded highest correlation pixel to the GCP making the GCP a seven-tuple to register the second 2D image to the 3D point set.

2. The method of claim 1 , further comprising comparing a ratio, of the highest correlation score to a second highest correlation score, to a second threshold value and discarding the second pixel location if the ratio is less than the second threshold value.

3. The method of claim 1 , wherein the first pixel is the center of the first image template.

4. The method of claim 1 , wherein the second pixel is the center of the second image template.

5. The method of claim 1 , further comprising warping the second image template using an affine transformation before identifying the second pixel.

6. The method of claim 5 , further comprising projecting the identified second pixel to an image space of the second pixel to determine the second pixel location.

7. A non-transitory machine-readable medium including instructions that, when executed by a machine, cause a machine to perform operations for determining a multi-image three-dimensional (3D) ground control point (GCP), the operations comprising:
extracting, from a first two-dimensional (2D) image including image data of a first geographical region, a first image template, the first image template including a contiguous subset of pixels from the first 2D image and a first pixel of the contiguous subset of pixels of the first 2D image indicated by the GCP, wherein the GCP is a five-tuple that associates a point in a three-dimensional (3D) point set with the first pixel of the first image;
predicting a first pixel location of the first pixel of the GCP in a second 2D image, the second 2D image including image data of a second geographical region overlapping with the first geographical region;
extracting, from the second 2D image, a second image template, the second image template including a contiguous subset of pixels from the second 2D image and a second pixel of the contiguous subset of pixels of the second 2D image corresponding to the first pixel location;
determining respective correlation scores, using a normalized cross-correlation, for the first image template centered at a variety of pixels of the contiguous subset of pixels of the second image template;
comparing a highest correlation score of the correlation scores to a first threshold value;
discarding a location of the second pixel responsive to determining a ratio is less than the first threshold value;
identifying an undiscarded highest correlation pixel of the second 2D image corresponding to a remaining highest correlation score; and
adding a pixel location of the undiscarded highest correlation pixel to the GCP making the GCP a seven-tuple to register the second 2D image to the 3D point set.

8. The non-transitory machine-readable medium of claim 7 , wherein the operations further comprise comparing a ratio, of the highest correlation score to a second highest correlation score, to a second threshold value and discarding the second pixel location if the ratio is less than the second threshold value.

9. The non-transitory machine-readable medium of claim 7 , wherein the first pixel is the center of the first image template.

10. The non-transitory machine-readable medium of claim 7 , wherein the second pixel is the center of the second image template.

11. The non-transitory machine-readable medium of claim 7 , wherein the operations further comprise warping the second image template using an affine transformation before identifying the second pixel.

12. The non-transitory machine-readable medium of claim 11 , wherein the operations further comprise projecting the identified second pixel to an image space of the second pixel to determine the second pixel location.

13. A system comprising:
a memory including image data of first and second images of a geographical region stored thereon; and
processing circuitry coupled to the memory, the processing circuitry configured to perform steps comprising:
extracting, from a first two-dimensional (2D) image including image data of a first geographical region, a first image template, the first image template including a contiguous subset of pixels from the first 2D image and a first pixel of the contiguous subset of pixels of the first 2D image indicated by the GCP, wherein the GCP is a five-tuple that associates a point in the 3D point set with the first pixel of the first image;
predicting a first pixel location of the first pixel of the GCP in a second 2D image, the second 2D image including image data of a second geographical region overlapping with the first geographical region;
extracting, from the second 2D image, a second image template, the second image template including a contiguous subset of pixels from the second 2D image and a second pixel of the contiguous subset of pixels of the second 2D image corresponding to the first pixel location;
determining respective correlation scores, using a normalized cross-correlation, for the first image template centered at a variety of pixels of the contiguous subset of pixels of the second image template;
comparing a highest correlation score of the correlation scores to a first threshold value;
discarding a location of the second pixel responsive to determining the ratio is less than the first threshold value;
identifying an undiscarded highest correlation pixel of the second 2D image corresponding to a remaining highest correlation score; and
adding a pixel location of the undiscarded highest correlation pixel to the GCP making the GCP a seven-tuple to register the second 2D image to the 3D point set.

14. The system of claim 13 , wherein the processing circuitry is further configured to compare a ratio, of the highest correlation score to a second highest correlation score, to a second threshold value and discarding the second pixel location if the ratio is less than the second threshold value.

Assignees

• Raytheon Co

Inventors

• Ely Richard W
• Verret Jody D

Classifications

• G06T2207/30244

  Camera pose · CPC title

• G06T2207/10032

  Satellite or aerial image; Remote sensing · CPC title

• G01C11/02

  Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures · CPC title

• G06T2207/30181

  Earth observation · CPC title

• G06T7/74 Primary

  involving reference images or patches · CPC title