Roof scan using unmanned aerial vehicle

Patent metadata

Field	Value
Publication number	US-12379731-B2
Application number	US-202418435299-A
Country	US
Kind code	B2
Filing date	Feb 7, 2024
Priority date	Oct 28, 2019
Publication date	Aug 5, 2025
Grant date	Aug 5, 2025

Abstract

Described herein are systems for roof scan using an unmanned aerial vehicle. For example, some methods include capturing, using an unmanned aerial vehicle, an overview image of a roof of a building from above the roof; presenting a suggested bounding polygon overlaid on the overview image to a user; determining a bounding polygon based on the suggested bounding polygon and user edits; based on the bounding polygon, determining a flight path including a sequence of poses of the unmanned aerial vehicle with respective fields of view at a fixed height that collectively cover the bounding polygon; fly the unmanned aerial vehicle to a sequence of scan poses with horizontal positions matching respective poses of the flight path and vertical positions determined to maintain a consistent distance above the roof; and scanning the roof from the sequence of scan poses to generate a three-dimensional map of the roof.

First claim

What is claimed is:

1. A method, comprising:
determining a two-dimensional polygon as a convex hull of points, wherein the points correspond to a three-dimensional map of a roof to inspect using an unmanned aerial vehicle;
outputting, to a graphical user interface for display at a user device, the two-dimensional polygon overlaid on an overview image of the roof;
determining an edited two-dimensional polygon in an image plane of the overview image based on a user edit of the two-dimensional polygon within the graphical user interface;
determining, based on the edited two-dimensional polygon, a flight path including a sequence of poses representing positions and orientations for the unmanned aerial vehicle as the unmanned aerial vehicle performs an inspection of the roof; and controlling the unmanned aerial vehicle to perform the inspection of the roof.

2. The method of claim 1 , comprising:
generating a facet suggestion based on the three-dimensional map, wherein the points of the three-dimensional map correspond to the facet suggestion.

3. The method of claim 2 , wherein generating the facet suggestion based on the three-dimensional map comprises:
searching for a largest expanse of coplanar points in the three-dimensional map with a low ratio of outlier points; and
fitting a plane to the largest expanse of coplanar points.

4. The method of claim 2 , wherein generating the facet suggestion based on the three-dimensional map comprises:
obtaining a user selection, within the graphical user interface, of a facet appearing in the overview image.

5. The method of claim 1 , wherein determining the two-dimensional polygon as the convex hull of the points comprises:
computing the convex hull to obtain the two-dimensional polygon.

6. The method of claim 5 , wherein computing the convex hull to obtain the two-dimensional polygon comprises:
identifying a convex edge of the two-dimensional polygon;
determining that removal of the convex edge from the two-dimensional polygon would result in an increase of area of the two-dimensional polygon; and
simplifying the two-dimensional polygon by removing the convex edge and extending adjacent edges to an intersection point.

7. The method of claim 6 , wherein determining that the removal of the convex edge from the two-dimensional polygon would result in the increase of area of the two-dimensional polygon comprises:
determining a perimeter increase resulting from the removal of the convex edge from the two-dimensional polygon; and
determining whether the perimeter increase is greater than or equal to a threshold,
wherein simplifying the two-dimensional polygon by removing the convex edge and extending the adjacent edges to the intersection point comprises:
simplifying the two-dimensional polygon based on a determination that the perimeter increase is greater than or equal to the threshold.

8. The method of claim 1 , wherein determining the flight path comprises:
determining a facet by mapping the edited two-dimensional polygon to a subset of points of the three-dimensional map; and
determining a pose for a scan plan of the roof based on the facet, wherein the flight path corresponds to the scan plan.

9. The method of claim 8 , wherein determining the facet by mapping the edited two-dimensional polygon to the subset of points of the three-dimensional map comprises:
fitting a plane to the subset of points; and
computing a convex hull of the subset of points as projected onto the plane.

10. An apparatus, comprising:
a memory; and
a processor configured to execute instructions stored in the memory to:
determine a two-dimensional polygon as a convex hull of points, wherein the points correspond to a three-dimensional map of a roof to inspect;
determine a flight path including a sequence of poses representing positions and orientations for an unmanned aerial vehicle as the unmanned aerial vehicle performs an inspection of the roof based on a user edit, within a graphical user interface to which the two-dimensional polygon is overlaid on an overview image of the roof, of the two-dimensional polygon; and control the unmanned aerial vehicle to perform the inspection of the roof.

11. The apparatus of claim 10 , wherein an edited two-dimensional polygon is determined in an image plane of the overview image based on the user edit and the flight path is determined based on the edited two-dimensional polygon.

12. The apparatus of claim 10 , wherein a convex edge of the two-dimensional polygon is removed to simplify the two-dimensional polygon before it is overlaid on the overview image of the roof within the graphical user interface.

13. The apparatus of claim 10 , wherein the points correspond to a facet suggestion generated based on one of a plane fit to an expanse of coplanar points in the three-dimensional map or a user selection of a facet appearing in the overview image.

14. A system, comprising:
a user device configured to output, for display, a graphical user interface including an overview image of a roof to inspect; and
an unmanned aerial vehicle configured to determine a two-dimensional polygon as a convex hull of points of a three-dimensional map of the roof, output the two-dimensional polygon overlaid on the overview image of the roof to the graphical user interface, and determine, based on a user edit of the two-dimensional polygon within the graphical user interface, a flight path including a sequence of poses representing positions and orientations for the unmanned aerial vehicle as the unmanned aerial vehicle performs an inspection of the roof, wherein the unmanned aerial vehicle is controlled to perform the inspection of the roof.

15. The system of claim 14 , wherein the two-dimensional polygon according to the user edit is mapped to a subset of points of the three-dimensional map to determine a facet and the flight path is determined based on the facet.

16. The system of claim 14 , wherein the two-dimensional polygon is simplified to remove a convex edge prior to the output of the two-dimensional polygon to the graphical user interface.

17. The system of claim 14 , wherein the user device captures data indicative of the user edit and transmits the data to the unmanned aerial vehicle for processing.

18. The system of claim 14 , wherein the overview image is captured using one or more image sensors of the unmanned aerial vehicle.

19. The system of claim 14 , wherein the graphical user interface indicates a progress of execution of the flight path during the inspection of the roof.

20. The system of claim 14 , wherein the unmanned aerial vehicle is configured to determine a pose of the sequence of poses based on a facet determined based on a mapping of the edited two-dimensional polygon to a subset of points of the three-dimensional map.

Assignees

• Skydio Inc

Inventors

• Henry Peter
• Zhu Jack
• Richman Brian
• Zheng Harrison
• Martirosyan Hayk
• Donahoe Matthew
• Bachrach Abraham Galton
• Bry Adam

Classifications

• B64C39/024 Primary

  of the remote controlled vehicle type, i.e. RPV · CPC title

• G05D1/223

  Command input arrangements on the remote controller, e.g. joysticks or touch screens · CPC title

• G05D1/224

  Output arrangements on the remote controller, e.g. displays, haptics or speakers · CPC title

• G05D1/606

  Compensating for or utilising external environmental conditions, e.g. wind or water currents (station keeping G05D1/611) · CPC title

• G08G5/80

  Anti-collision systems · CPC title