Automation methods for UAV perching on pipes

Patent metadata

Field	Value
Publication number	US-11235823-B2
Application number	US-201916696085-A
Country	US
Kind code	B2
Filing date	Nov 26, 2019
Priority date	Nov 29, 2018
Publication date	Feb 1, 2022
Grant date	Feb 1, 2022

Abstract

An unmanned aerial vehicle (UAV) autonomously perching on a curved surface from a starting position is provided. The UAV includes: a 3D depth camera configured to capture and output 3D point clouds of scenes from the UAV including the curved surface; a 2D LIDAR system configured to capture and output 2D slices of the scenes; and a control circuit. The control circuit is configured to: control the depth camera and the LIDAR system to capture the 3D point clouds and the 2D slices, respectively, of the scenes; input the captured 3D point clouds from the depth camera and the captured 2D slices from the LIDAR system; autonomously detect and localize the curved surface using the captured 3D point clouds and 2D slices; and autonomously direct the UAV from the starting position to a landing position on the curved surface based on the autonomous detection and localization of the curved surface.

First claim

What is claimed is:

1. An unmanned aerial vehicle (UAV) for autonomously perching on a cylindrically curved surface from a starting position away from the cylindrically curved surface, the UAV comprising:
a three-dimensional (3D) depth camera configured to capture and output 3D point clouds of scenes from the UAV comprising the cylindrically curved surface;
a two-dimensional (2D) light detection and ranging (LIDAR) system having a one-dimensional (1D) field of view (FOV), in contrast to a 3D LIDAR system having a 2D FOV, the 2D LIDAR system being configured to capture and output 2D slices of the scenes;
a perching mechanism configured to mechanically perch the UAV on the cylindrically curved surface; and
a control circuit configured to:
control the depth camera and the 2D LIDAR system to capture the 3D point clouds and the 2D slices, respectively, of the scenes;
input the captured 3D point clouds from the depth camera and the captured 2D slices from the 2D LIDAR system;
autonomously detect and localize the cylindrically curved surface using the captured 3D point clouds and the captured 2D slices;
autonomously direct the UAV from the starting position to a landing position on the cylindrically curved surface based on the autonomous detection and localization of the cylindrically curved surface; and
autonomously control the UAV to perch the UAV on the cylindrically curved surface at the landing position using the perching mechanism.

2. The UAV of claim 1 , wherein the control circuit is further configured to:
use the captured 3D point clouds to perform the autonomous detection and localization from the starting position while autonomously directing the UAV to the landing position until the UAV reaches a proximity of the cylindrically curved surface; and
switch from using the 3D point clouds to using the captured 2D slices to perform the autonomous detection and localization once the UAV reaches the proximity of the cylindrically curved surface.

3. The UAV of claim 2 , wherein the control circuit is further configured to:
autonomously direct the UAV to an alignment position within the proximity of the cylindrically curved surface prior to the perching, the alignment position being where the UAV is aligned with respect to the landing position; and
use the captured 2D slices to autonomously direct the UAV to move directly from the alignment position to the landing position.

4. The UAV of claim 1 , wherein the control circuit is further configured to autonomously detect and localize the cylindrically curved surface by fusing the 3D point clouds and the 2D slices, the fusing comprising:
using one of the captured 3D point clouds to perform a first pass of the autonomous detection and localization of the cylindrically curved surface;
capturing a corresponding one of the 2D slices using the detected and localized cylindrically curved surface from the first pass; and
using the corresponding one of the 2D slices to perform a second pass of the autonomous detection and localization of the cylindrically curved surface.

5. The UAV of claim 1 , further comprising an inertial measurement unit (IMU) configured to estimate the pose of the depth camera, wherein the control circuit is further configured to use the estimated pose of the depth camera between capturing a first one of the 3D point clouds and capturing a second one of the 3D point clouds in order to predict the location of the cylindrically curved surface in the second one of the 3D point clouds from the detected and localized cylindrically curved surface in the first one of the 3D point clouds.

6. The UAV of claim 1 , wherein the control circuit is further configured to:
use the captured 3D point clouds to autonomously detect and localize the cylindrically curved surface; and
control the 2D LIDAR system to capture the 2D slices normal to the detected and localized cylindrically curved surface.

7. The UAV of claim 6 , wherein the cylindrically curved surface is part of a cylinder and the control circuit is further configured to control the 2D LIDAR system to capture the 2D slices normal to the central axis of the cylinder.

8. The UAV of claim 1 , wherein the control circuit is further configured to use a random sample consensus (RANSAC) approach to autonomously detect and localize the cylindrically curved surface.

9. The UAV of claim 1 , wherein the control circuit is further configured to:
use the captured 3D point clouds or the captured 2D slices or both the captured 3D point clouds and the captured 2D slices to detect one or more obstacles on a flight path of the UAV from the starting position to the landing position; and
autonomously redirect the UAV to avoid the one or more obstacles on the flight path.

10. The UAV of claim 1 , wherein the cylindrically curved surface is ferromagnetic and the perching mechanism comprises magnetic legs configured to magnetically attach to the ferromagnetic cylindrically curved surface during the perching and remain magnetically attached to the ferromagnetic cylindrically curved surface after the perching.

11. A method of autonomously perching an unmanned aerial vehicle (UAV) on a cylindrically curved surface from a starting position away from the cylindrically curved surface, the method comprising:
capturing and outputting, using a three-dimensional (3D) depth camera attached to the UAV, 3D point clouds of scenes from the UAV comprising the cylindrically curved surface;
capturing and outputting, using a two-dimensional (2D) light detection and ranging (LIDAR) system attached to the UAV, 2D slices of the scenes, the 2D LIDAR system having a one-dimensional (1D) field of view (FOV), in contrast to a 3D LIDAR system having a 2D FOV;
mechanically perching the UAV on the cylindrically curved surface using a perching mechanism attached to the UAV;
controlling the depth camera and the 2D LIDAR system to capture the 3D point clouds and the 2D slices, respectively, of the scenes;
inputting the captured 3D point clouds from the depth camera and the captured 2D slices from the 2D LIDAR system;
autonomously detecting and localizing the cylindrically curved surface using the captured 3D point clouds and the captured 2D slices;
autonomously directing the UAV from the starting position to a landing position on the cylindrically curved surface based on the autonomous detection and localization of the cylindrically curved surface; and
autonomously controlling the UAV to perch the UAV on the cylindrically curved surface at the landing position using the perching mechanism.

12. The method of claim 11 , further comprising:
using the captured 3D point clouds to perform the autonomous detection and localization from the starting position while autonomously directing the UAV to the landing position until the UAV reaches a proximity of the cylindrically curved surface; and
switching from using the 3D point clouds to using the captured 2D slices to perform the autonomous detection and localization once the UAV reaches the proximity of the cylindrically curved surface.

13. The method of claim 12 , further comprising:
autonomously directing the UAV to an alignment position within the proximity of the cylindrically curved surface prior to the perching, where the UAV is aligned with respect to the landing position; and
using the captured 2D slices to autonomously direct the UAV to move directly from the alignment position to the landing position.

14. The method of claim 11 , further comprising autonomously detecting and localizing the cylindrically curved surface by fusing the 3D point clouds and the 2D slices, the fusing comprising:
using one

Assignees

• Saudi Arabian Oil Co
• Univ King Abdullah Sci & Tech

Inventors

• Abdelkader Mohamed
• Abdellatif Fadl
• Shamma Jeff S

Classifications

• B60B19/006 Primary

  Magnetic wheels · CPC title

• G06V20/17

  taken from planes or by drones · CPC title

• G06V20/13

  Satellite images · CPC title

• B62D57/024 Primary

  specially adapted for moving on inclined or vertical surfaces (endless-track vehicles for ascending or descending stairs B62D55/075; hand-carts with provision for travelling up or down stairs B62B5/02) · CPC title

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title