Structure scan using unmanned aerial vehicle

Patent metadata

Field	Value
Publication number	US-12097957-B2
Application number	US-202217890884-A
Country	US
Kind code	B2
Filing date	Aug 18, 2022
Priority date	Oct 28, 2019
Publication date	Sep 24, 2024
Grant date	Sep 24, 2024

Abstract

Described herein are systems and methods for structure scan using an unmanned aerial vehicle. For example, some methods include accessing a three-dimensional map of a structure; generating facets based on the three-dimensional map, wherein the facets are respectively a polygon on a plane in three-dimensional space that is fit to a subset of the points in the three-dimensional map; generating a scan plan based on the facets, wherein the scan plan includes a sequence of poses for an unmanned aerial vehicle to assume to enable capture, using image sensors of the unmanned aerial vehicle, of images of the structure; causing the unmanned aerial vehicle to fly to assume a pose corresponding to one of the sequence of poses of the scan plan; and capturing one or more images of the structure from the pose.

First claim

What is claimed is:

1. A method, comprising:
determining one or more polygons on one or more planes by, for at least one polygon of the one or more polygons, searching within a three-dimensional map of a structure for a largest expanse of coplanar points that minimizes outlier points and fitting a plane of the one or more planes to the largest expanse of coplanar points;
determining a scan plan for inspecting the structure using an unmanned aerial vehicle based on the one or more polygons; and
capturing one or more images of the structure using a camera of the unmanned aerial vehicle while the unmanned aerial vehicle moves according to the scan plan.

2. The method of claim 1 , wherein the three-dimensional map encodes a set of points in a three-dimensional space on surfaces of the structure, and wherein the set of points includes the coplanar points.

3. The method of claim 1 , wherein the scan plan includes a sequence of poses, and wherein capturing the one or more images of the structure comprises:
causing the unmanned aerial vehicle to assume the sequence of poses.

4. The method of claim 3 , wherein each pose of the sequence of poses corresponds to a polygon of the one or more polygons.

5. The method of claim 3 , wherein capturing the one or more images of the structure comprises:
adjusting a pose of the sequence of poses while the unmanned aerial vehicle moves according to the scan plan.

6. The method of claim 1 , wherein at least one polygon of the one or more polygons is determined based on an overview image of the structure.

7. The method of claim 6 , wherein the at least one polygon is determined as a convex hull of a subset of points projected into an image plane of the overview image.

8. The method of claim 6 , comprising:
modifying the at least one polygon based on user input obtained via a graphical user interface within which the overview image is presented.

9. The method of claim 1 , wherein the three-dimensional map is generated based on scan images captured using the camera of the unmanned aerial vehicle.

10. The method of claim 1 , wherein ones of the one or more polygons for which an image of the one or more images is captured are indicated within a coverage map.

11. An unmanned aerial vehicle, comprising:
a propulsion mechanism configured to fly the unmanned aerial vehicle for an inspection of a structure;
a camera configured to capture one or more images of the structure;
a memory storing instructions; and
a processor configured to execute the instructions to:
determine a scan plan for inspecting the structure based on one or more polygons, wherein the one or more polygons are defined on one or more planes by, for at least one polygon of the one or more polygons, searching within a three-dimensional map of the structure for a largest expanse of coplanar points that minimizes outlier points and fitting a plane of the one or more planes to the largest expanse of coplanar points;
cause the propulsion mechanism to fly the unmanned aerial vehicle according to the scan plan; and
cause the camera to capture the one or more images while the unmanned aerial vehicle flies according to the scan plan.

12. The unmanned aerial vehicle of claim 11 , wherein the processor is configured to execute the instructions to:
cause the camera to capture one or more initial scan images of the structure; and
generate the three-dimensional map using the one or more initial scan images.

13. The unmanned aerial vehicle of claim 11 , wherein the scan plan includes a sequence of poses, wherein each pose of the sequence of poses corresponds to a polygon of the one or more polygons, and wherein each of the one or more images is captured at a pose of the sequence of poses.

14. The unmanned aerial vehicle of claim 11 , wherein the processor is configured to execute the instructions to:
determine at least one polygon of the one or more polygons using input obtained via a graphical user interface within which information associated with the structure is presented.

15. The unmanned aerial vehicle of claim 11 , wherein each of the one or more images is captured at a consistent distance from the structure.

16. A system, comprising:
a remote controller device configured to obtain input via a graphical user interface displaying information associated with a structure; and
an unmanned aerial vehicle configured to capture one or more images of the structure while the unmanned aerial vehicle moves along a scan plan for the structure, wherein the scan plan is based on one or more polygons on one or more planes by, for at least one polygon of the one or more polygons, searching within a three-dimensional map of the structure for a largest expanse of coplanar points that minimizes outlier points and fitting a plane of the one or more planes to the largest expanse of coplanar points, and wherein at least one polygon of the one or more polygons is determined using the input.

17. The system of claim 16 , wherein the scan plan is determined at the remote controller device.

18. The system of claim 16 , wherein the scan plan is determined at the unmanned aerial vehicle.

19. The system of claim 16 , wherein the scan plan includes a sequence of poses at which to capture images of the one or more images, and wherein at least one pose of the sequence of poses corresponds to the at least one polygon.

20. The system of claim 16 , wherein, in response to a capture of an image of the one or more images at the unmanned aerial vehicle, the graphical user interface is updated at the remote controller device to indicate a coverage of a polygon associated with the image.

Assignees

• Skydio Inc

Inventors

• Henry Peter
• Zhu Jack
• Richman Brian
• Zheng Harrison
• Martirosyan Hayk
• Donahoe Matthew
• Bachrach Abraham Galton
• Bry Adam

Classifications

• B64C39/024 Primary

  of the remote controlled vehicle type, i.e. RPV · CPC title

• G05D1/223

  Command input arrangements on the remote controller, e.g. joysticks or touch screens · CPC title

• G05D1/224

  Output arrangements on the remote controller, e.g. displays, haptics or speakers · CPC title

• G05D1/606

  Compensating for or utilising external environmental conditions, e.g. wind or water currents (station keeping G05D1/611) · CPC title

• G08G5/80

  Anti-collision systems · CPC title