Structural modification detection using descriptor-based querying

Patent metadata

Field	Value
Publication number	US-12374065-B1
Application number	US-202217864848-A
Country	US
Kind code	B1
Filing date	Jul 14, 2022
Priority date	Jul 10, 2019
Publication date	Jul 29, 2025
Grant date	Jul 29, 2025

Abstract

Identifying a pre-existing three-dimensional (3D) model of a target structure includes receiving at least one two-dimensional (2D) image of a target physical structure; generating a predicted 3D model of the target structure based on the at least one 2D image; generating a search descriptor of the predicted 3D model; querying a data structure storing a plurality of pre-existing descriptors, where each pre-existing descriptor characterizes a previously constructed 3D model of an associated physical structure; and identifying at least one previously constructed 3D model that is substantially similar to the predicted 3D model based on a difference between the search descriptor and the plurality of pre-existing descriptors.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving at least one two-dimensional (2D) image of a building;
generating a predicted 3D model of the building based on the at least one 2D image;
generating a search descriptor of the predicted 3D model, wherein the search descriptor comprises a vector of binary values describing a presence or absence of structural features at specific locations on the building to represent the building geometry;
querying a data structure storing a plurality of pre-existing descriptors, wherein each pre-existing descriptor characterizes a previously constructed 3D model of an associated physical structure; and
identifying at least one previously constructed 3D model that is substantially similar to the predicted 3D model based on a difference between the search descriptor and the plurality of pre-existing descriptors.

2. The computer-implemented method of claim 1 , wherein generating the search descriptor of the predicted 3D model comprises compressing the predicted 3D model relative to a camera perspective.

3. The computer-implemented method of claim 2 , wherein the camera perspective comprises a top-down view.

4. The computer-implemented method of claim 1 , wherein the search descriptor comprises a set of bits is generated with an auto-encoder network, and wherein each bit in the vector of binary values represents the presence or absence of a specific structural feature of the associated physical structure.

5. The computer-implemented method of claim 1 , wherein the difference between the search descriptor and the plurality of pre-existing descriptors comprises a Hamming distance.

6. The computer-implemented method of claim 1 , further comprising mapping the at least one previously constructed 3D model to the predicted 3D model.

7. The computer-implemented method of claim 6 , further comprising modifying the at least one previously constructed 3D model according to a geometry of the predicted 3D model.

8. The computer-implemented method of claim 7 , wherein the geometry of the predicted 3D model comprises a structural modification of the building as captured by the at least one 2D image.

9. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving at least one two-dimensional (2D) image of a building;
generating a predicted 3D model of the building based on the at least one 2D image;
generating a search descriptor of the predicted 3D model, wherein the search descriptor comprises a vector of binary values describing a presence or absence of structural features at specific locations on the building to represent the building geometry;
querying a data structure storing a plurality of pre-existing descriptors, wherein each pre-existing descriptor characterizes a previously constructed 3D model of an associated physical structure; and
identifying at least one previously constructed 3D model that is substantially similar to the predicted 3D model based on a difference between the search descriptor and the plurality of pre-existing descriptors.

10. The non-transitory computer-readable medium of claim 9 , wherein generating the search descriptor of the predicted 3D model comprises compressing the predicted 3D model relative to a camera perspective.

11. The non-transitory computer-readable medium of claim 10 , wherein the camera perspective comprises a top-down view.

12. The non-transitory computer-readable medium of claim 9 , wherein the search descriptor comprises a set of bits is generated with an auto-encoder network, and wherein each bit in the vector of binary values represents the presence or absence of a specific structural feature of the associated physical structure.

13. The non-transitory computer-readable medium of claim 12 , wherein the building comprises a building, and each bit in the vector of binary values indicates the existence or absence of a specific structural feature of the building.

14. The non-transitory computer-readable medium of claim 9 , wherein the operations further comprise:
mapping the at least one previously constructed 3D model to the predicted 3D model; and
modifying the at least one previously constructed 3D model according to a geometry of the predicted 3D model, wherein the geometry of the predicted 3D model comprises a structural modification of the building as captured by the at least one 2D image.

15. A system comprising:
one or more processors;
one or more memory devices storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving at least one two-dimensional (2D) image of a building;
generating a predicted 3D model of the building based on the at least one 2D image;
generating a search descriptor of the predicted 3D model, wherein the search descriptor comprises a vector of binary values describing a presence or absence of structural features at specific locations on the building to represent the building geometry;
querying a data structure storing a plurality of pre-existing descriptors, wherein each pre-existing descriptor characterizes a previously constructed 3D model of an associated physical structure; and
identifying at least one previously constructed 3D model that is substantially similar to the predicted 3D model based on a difference between the search descriptor and the plurality of pre-existing descriptors.

16. The system of claim 15 , wherein generating the search descriptor of the predicted 3D model comprises compressing the predicted 3D model relative to a camera perspective.

17. The system of claim 15 , wherein the search descriptor comprises a set of bits is generated with an auto-encoder network, and wherein each bit in the vector of binary values represents the presence or absence of a specific structural feature of the associated physical structure.

18. The system of claim 15 , wherein the difference between the search descriptor and the plurality of pre-existing descriptors comprises a Hamming distance.

19. The system of claim 15 , wherein the operations further comprise:
mapping the at least one previously constructed 3D model to the predicted 3D model; and
modifying the at least one previously constructed 3D model according to a geometry of the predicted 3D model, wherein the geometry of the predicted 3D model comprises a structural modification of the building as captured by the at least one 2D image.

Assignees

• Hover Inc

Inventors

• Rethage Dario
• Scott Brandon
• Upendran Manish
• Lacasse Randy
• Sanjurjo Javier Cambon

Classifications

• G06V2201/10

  Recognition assisted with metadata · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V20/653

  by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces · CPC title

• G06F16/53

  Querying · CPC title