Systems and methods for estimating feature locations based on image analysis and homography

Patent metadata

Field	Value
Publication number	US-12456209-B2
Application number	US-202017638773-A
Country	US
Kind code	B2
Filing date	Aug 27, 2020
Priority date	Aug 28, 2019
Publication date	Oct 28, 2025
Grant date	Oct 28, 2025

Abstract

Techniques are described for identifying correspondences between images to generate a fundamental matrix for the camera positions related to the images. The resultant fundamental matrix enables epipolar geometry to correlate common features among the images. Correspondences are identified by confirming feature matches across images by applying a homography to data representing features across images. Further techniques are described herein for generating a representation of a boundary of a feature of a structure based on a digital image. In one or more embodiments, generating a representation of a boundary of a particular feature in a digital image comprises determining a portion of the image that corresponds to the structure, and determining a portion of the image that corresponds to the particular feature. One more vanishing points are associated with the portion of the image corresponding to the particular feature. The one or more vanishing points are used to generate a set of bounding lines for the particular feature, based on which the boundary indicator for the feature is generated.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
receiving a first image comprising data for a set of representations of co-planar rectangular features on a surface having a first orientation; receiving a second image comprising data for the set of representations of co-planar rectangular features on the surface having a second orientation; selecting, from the set of representations on the surface having the first orientation, a first representation, wherein the set of representations includes one or more first additional representations on the surface having the first orientation in addition to the first representation; selecting, from the set of representations on the surface having the second orientation, a second representation, wherein the set of representations includes one or more second additional representations on the surface having the second orientation in addition to the second representation; generating a homography aligning the first representation to the second representation; estimating locations of the first additional representations on the surface having the second orientation based on the homography; selecting an inlier pair formed from a first inlier and a second inlier based on the estimated locations, the inlier pair selected from the first additional representations and the second additional representations and the inlier pair being determined to match; and generating an adjusted homography collectively aligning the first representation and the first inlier to the second representation and the second inlier.

2 . The method of claim 1 , further comprising generating a fundamental matrix between the first and second images based on the homography.

3 . The method of claim 2 , further comprising:
drawing an epipolar line in the second image based on the fundamental matrix with the first image; and correlating at least one feature in the first image in the second image.

4 . The method of claim 3 , further comprising estimating at least one camera position based on the correlated features.

5 . The method of claim 4 , further comprising reconstructing a three dimensional model of the correlated features based on the estimated camera position.

6 . The method of claim 1 , further comprising validating the homography based on a comparison between the estimated locations of one or more of the first additional representations relative to the second additional representations, wherein the first additional representations and the second additional representations are associated with same co-planar rectangular features.

7 . The method of claim 6 , wherein validating further comprises detecting non-alignment for all estimated locations of the first additional representations and at least one of the second additional representations.

8 . The method of claim 6 , wherein validating further comprises detecting an error below a threshold for a position of the estimated locations of at least one of the first additional representations and at least one of the second additional representations.

9 . The method of claim 1 , further comprising generating a fundamental matrix between the first and second images based on the adjusted homography.

10 . The method of claim 6 , wherein validating further comprises detecting that a particular feature representation, of the first additional representations, is an inlier based on determining that the estimated location of the particular feature representation is within a threshold number of pixels from locations of at least one of the second additional representations.

11 . The method of claim 10 , further comprising:
determining that one or more other feature representations, other than the particular feature representation of the first additional representations, are mis-aligned; and in response to determining that the one or more other feature representations of the first additional representations, are mis-aligned, adjusting the homography based on an estimated position of at least one of the one or more other feature representations.

12 . The method of claim 11 , further comprising generating a fundamental matrix between the first and second images based on the adjusted homography.

13 . A computer-readable storage medium comprising instructions which, when executed by a computer system, cause the computer system to:
receive a first image comprising data for a set of representations of co-planar rectangular features on a surface having a first orientation; receive a second image comprising data for representations of the set of co-planar rectangular features on the surface having a second orientation; select, from the set of representations on the surface having the first orientation, a first representation, wherein the set of representations includes one or more first additional representations on the surface having the first orientation in addition to the first representation; select, from the set of representations on the surface having the second orientation, a second representation, wherein the set of representations includes one or more second additional representations on the surface having the second orientation in addition to the second representation; generate a homography aligning the first representation to the second representation; estimate locations of the first additional representations on the surface in the second orientation based on the homography; select an inlier pair formed from a first inlier and a second inlier based on the estimated locations, the inlier pair selected from the first additional representations and the second additional representations and the inlier pair being determined to match; and generate an adjusted homography collectively aligning the first representation and the first inlier to the second representation and the second inlier.

14 . The computer-readable storage medium of claim 13 , further comprising instructions that, when executed by the computer system, cause the computer system to generate a fundamental matrix between the first and second images based on the homography.

15 . The computer-readable storage medium of claim 14 , further comprising instructions that, when executed by the computer system, cause the computer system to:
draw an epipolar line in the second image based on the fundamental matrix with the first image; and correlate at least one feature in the first image in the second image.

16 . The computer-readable storage medium of claim 15 , further comprising instructions that, when executed by the computer system, cause the computer system to estimate at least one camera position based on the correlated features.

17 . The computer-readable storage medium of claim 16 , further comprising instructions that, when executed by the computer system, cause the computer system to reconstruct a three dimensional model of the correlated features based on the estimated camera position.

18 . The computer-readable storage medium of claim 13 , further comprising instructions that, when executed by a computer system, cause the computer system to validate the homography based on a comparison between the estimated locations of one or more of the first additional representations relative to the second additional representations, wherein the first additional representations and the second additional representations are associated with same co-planar rectangular features.

19 . The computer-readable storage medium of claim 18 , wherein the instructions, when executed by the com

Assignees

• Hover Inc

Inventors

• Shree Atulya
• Jia Kai
• Xiong Zhiyao
• Chow Siu Fai
• Phan Raymond
• Li Panfeng
• Curro Domenico

Classifications

• G06T7/33

  using feature-based methods · CPC title

• G06T7/73

  using feature-based methods · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T7/543 Primary

  from line drawings · CPC title

• G06T7/579 Primary

  from motion · CPC title