Structure scan using unmanned aerial vehicle

Patent metadata

Field	Value
Publication number	US-11455894-B2
Application number	US-202016896066-A
Country	US
Kind code	B2
Filing date	Jun 8, 2020
Priority date	Oct 28, 2019
Publication date	Sep 27, 2022
Grant date	Sep 27, 2022

Abstract

Described herein are systems and methods for structure scan using an unmanned aerial vehicle. For example, some methods include accessing a three-dimensional map of a structure; generating facets based on the three-dimensional map, wherein the facets are respectively a polygon on a plane in three-dimensional space that is fit to a subset of the points in the three-dimensional map; generating a scan plan based on the facets, wherein the scan plan includes a sequence of poses for an unmanned aerial vehicle to assume to enable capture, using image sensors of the unmanned aerial vehicle, of images of the structure; causing the unmanned aerial vehicle to fly to assume a pose corresponding to one of the sequence of poses of the scan plan; and capturing one or more images of the structure from the pose.

First claim

What is claimed is:

1. An unmanned aerial vehicle comprising:
a propulsion mechanism,
one or more image sensors, and
a processing apparatus, wherein the processing apparatus is configured to:
access a three-dimensional map of a structure, wherein the three-dimensional map encodes a set of points in three-dimensional space on surfaces of the structure;
generate, using the three-dimensional map, a scan plan including a sequence of poses for the unmanned aerial vehicle to assume to capture images of the structure by:
generating one or more facets based on coplanar points of the set of points encoded in the three-dimensional space by the three-dimensional map, wherein a given facet of the one or more facets is a polygon on a plane in three-dimensional space fit to a subset of the points in the three-dimensional map; and
generating the scan plan based on the one or more facets;

control the propulsion mechanism to cause the unmanned aerial vehicle to fly to assume a pose corresponding to one of the sequence of poses of the scan plan; and
capture, using the one or more image sensors, one or more images of the structure from the pose.

2. The unmanned aerial vehicle of claim 1 , wherein the processing apparatus is configured to:
capture, using the one or more image sensors, an overview image of the structure;
generate a facet suggestion based on the three-dimensional map; and
determine a two-dimensional polygon as a convex hull of a subset of points of the three-dimensional map, the subset of points corresponding to the facet suggestion, as projected into an image plane of the overview image.

3. The unmanned aerial vehicle of claim 2 , wherein the processing apparatus is configured to:
present the two-dimensional polygon overlaid on the overview image;
determine an edited two-dimensional polygon in the image plane of the overview image based on data indicating a user edit of the two-dimensional polygon; and
determine one of the one or more facets based on the edited two-dimensional polygon.

4. The unmanned aerial vehicle of claim 3 , wherein the processing apparatus is configured to:
prior to presenting the two-dimensional polygon overlaid on the overview image, simplify the two-dimensional polygon by removing a convex edge from the two-dimensional polygon and extending edges of the two-dimensional polygon adjacent to the convex edge to a point at which the extended edges intersect each other.

5. The unmanned aerial vehicle of claim 4 , wherein the processing apparatus is configured to:
check that removal of the convex edge increases area of the two-dimensional polygon by an amount less than a threshold.

6. The unmanned aerial vehicle of claim 4 , wherein the processing apparatus is configured to:
check that removal of the convex edge increases perimeter of the two-dimensional polygon by an amount less than a threshold.

7. The unmanned aerial vehicle of claim 1 , wherein the sequence of poses of the scan plan are for orthographic imaging of each of the one or more facets.

8. The unmanned aerial vehicle of claim 1 , wherein the one or more images sensors are configured to support stereoscopic imaging used to provide range data, and wherein the processing apparatus is configured to:
control the propulsion mechanism to cause the unmanned aerial vehicle to fly to a vicinity of the structure; and
scan the structure using the one or more image sensors to generate the three-dimensional map.

9. The unmanned aerial vehicle of claim 1 , wherein the processing apparatus is configured to:
capture, using the one or more image sensors, an overview image of the structure;
present, to a user, a graphical representation of the scan plan overlaid on the overview image; and
receive an indication of an approval of the scan plan from the user.

10. The unmanned aerial vehicle of claim 1 , wherein the processing apparatus is configured to:
detect, while flying between poses in the sequence of poses of the scan plan, an obstacle, wherein the detection is performed based on images captured using the one or more image sensors; and
dynamically adjust a pose of the sequence of poses of the scan plan to avoid the obstacle.

11. The unmanned aerial vehicle of claim 1 , wherein the processing apparatus is configured to:
detect, while flying between poses in the sequence of poses of the scan plan, a deviation of points on a surface of the structure from one of the one or more facets, wherein the detection is performed based on images captured using the one or more image sensors; and
dynamically adjust a pose of the sequence of poses of the scan plan to adapt to the deviation and maintain a consistent distance for image capture.

12. The unmanned aerial vehicle of claim 1 , wherein the processing apparatus is configured to:
generate a coverage map of the one or more facets indicating which of the one or more facets have been successfully imaged during execution of the scan plan; and
present the coverage map.

13. The unmanned aerial vehicle of claim 1 , wherein the processing apparatus is configured to:
determine area estimates for each of the one or more facets; and
present a data structure including the one or more facets, the area estimates of each of the one or more facets, and images of the structure captured during execution of the scan plan.

14. The unmanned aerial vehicle of claim 1 , wherein the structure is a roof of a building, a bridge, or a building under construction.

15. A method comprising:
accessing a three-dimensional map of a structure, wherein the three-dimensional map encodes a set of points in three-dimensional space on surfaces of the structure;
generating one or more facets based on coplanar points of the set of points encoded in the three-dimensional space by the three-dimensional map, wherein a given facet of the one or more facets is a polygon on a plane in three-dimensional space fit to a subset of the points in the three-dimensional map; and
generating a scan plan based on the one or more facets, wherein the scan plan includes a sequence of poses for an unmanned aerial vehicle to assume to enable capture, using one or more image sensors of the unmanned aerial vehicle, of images of the structure at a consistent distance from each of the one or more facets.

16. The method of claim 15 , comprising:
controlling a propulsion mechanism of an unmanned aerial vehicle to cause the unmanned aerial vehicle to fly to assume a pose corresponding to one of the sequence of poses of the scan plan; and
capturing, using the one or more image sensors, one or more images of the structure from the pose.

17. The method of claim 16 , wherein the one or more images sensors are configured to support stereoscopic imaging used to provide range data, and the method comprises:
controlling the propulsion mechanism to cause the unmanned aerial vehicle to fly to a vicinity of the structure; and
scanning the structure using the one or more image sensors to generate the three-dimensional map.

18. The method of claim 16 , comprising:
detecting, while flying between poses in the sequence of poses of the scan plan, an obstacle, wherein the detection is performed based on images captured using the one or more image sensors; and
dynamically adjusting a pose of the sequence of poses of the scan plan to avoid the obstacle.

19. The method of claim 16 , comprising:
detecting, while flying between poses in the sequence of poses of the scan plan, a deviation of points on a surface of the structure from one of the one or more fa

Assignees

• Skydio Inc

Inventors

• Henry Peter
• Zhu Jack
• Richman Brian
• Zheng Harrison
• Martirosyan Hayk
• Donahoe Matthew
• Bachrach Abraham Galton
• Bry Adam

Classifications

• G06V20/17

  taken from planes or by drones · CPC title

• G06V20/647

  by matching two-dimensional images to three-dimensional objects · CPC title

• B64U2201/20

  Remote controls · CPC title

• B64U2201/00

  UAVs characterised by their flight controls · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title