Tunnel defect detecting method and system using unmanned aerial vehicle

Patent metadata

Field	Value
Publication number	US-12505525-B2
Application number	US-202217580935-A
Country	US
Kind code	B2
Filing date	Jan 21, 2022
Priority date	May 25, 2021
Publication date	Dec 23, 2025
Grant date	Dec 23, 2025

Abstract

Tunnel defect detecting method and system using unmanned aerial vehicle (UAV) are provided, and the UAV is equipped with a light-emitting diode (LED) module, a camera, a laser radar, an ultrasonic distance meter and an inertial measurement unit (IMU). The method includes: collecting images in a tunnel based on the LED module and the camera to obtain a training image set; training by using the training image set to obtain a defect detecting model, collecting real-time tunnel images, detecting suspected defects to the real-time tunnel images by the defect detecting model, obtaining pose information of the UAV based on the camera, the laser radar, the ultrasonic distance meter and the IMU to control the UAV to hover. The method can realize accurate pose estimation and defect detection in the tunnel with no GPS signals and highly symmetrical inside.

First claim

What is claimed is:

1 . A tunnel defect detecting method using an unmanned aerial vehicle (UAV), wherein the UAV is equipped with a light-emitting diode (LED) module, a camera, a laser radar, an ultrasonic distance meter and an inertial measurement unit (IMU); and the tunnel defect detecting method comprises:
S1, collecting a plurality of original images of tunnel defects in a tunnel based on the LED module and the camera to obtain an original image set, and pre-processing the original image set to obtain a training image set; S2, constructing a neural network model, and training the neural network model by using the training image set to obtain a defect detecting model; S3, acquiring pose information of the UAV based on the camera, the laser radar, the ultrasonic distance meter and the IMU, collecting real-time tunnel images based on the LED module and the camera while the UAV is flying in the tunnel, and detecting suspected defects in the real-time tunnel images by using the defect detecting model; wherein if the suspected defects are detected, executing S4; otherwise, continuing the flying of the UAV, and repeating S3; and S4, making the UAV to hover, collecting suspected defect images based on the LED module and the camera, using the defect detecting model to detect defects of the suspected defect images to obtain a defect detection result, recording position information of the UAV and the defect detection result, and then continuing the flying of the UAV, and executing S3; the acquiring pose information of the UAV based on the camera, the laser radar, the ultrasonic distance meter and the IMU, comprises: obtaining UAV motion information measured by the IMU, and obtaining first position information of the UAV based on the UAV motion information; obtaining UAV flight images collected by the camera being a binocular camera, pre-processing the UAV flight images to obtain pre-processed UAV flight images and then performing feature point tracking onto the pre-processed UAV flight images by an optical flow method to solve second position information of the UAV; wherein the binocular camera acquires left and right images during the UAV flight as follows: firstly, using histogram to equalize and enhance the image contrast of the two images, and using the adaptive threshold method to segment the images to obtain the black-and-white images; then, detecting the Fast feature points of the black-and-white images, tracking the feature points by using KLT (Kanade-Lucas-Tomasi Tracking) optical flow, eliminate the wrong tracking points by using RANSAC (Random Sample Consensus) algorithm, and tracking the optical flow of the left and right images to obtain the depth of feature points as follows:
depth=Bf x /d;
wherein, B represents the baseline length of left and right cameras, f x represents the focal length and d represents the parallax; obtaining radar point cloud data collected by the laser radar, and obtaining third position information of the UAV based on the radar point cloud data; scanning the surrounding environment by the laser radar, build the 2D SLAM in real-time, using Hector-SLAM algorithm when the original data of the laser radar matches the map currently built, using Gauss-Newton method to obtain the most probable pose of the UAV in the map at that moment as follows:
ξ*=argmin ξ Σ i=1 n [1− M ( S i (ξ))] 2 ;

wherein, ξ=(x, y, θ) T represents the pose information, S i (ξ) represents the representation of endpoint coordinates of the laser radar in the world coordinate system, and M(S i (ξ)) represents the map value of the coordinate point S i (ξ); obtaining absolute height information of the UAV based on measurement of the ultrasonic distance meter; and taking the first position information as predicted values, taking the second position information, the third position information and the absolute height information as observed values, and fusing the predicted values and the observed values to obtain the pose information of the UAV; wherein obtaining the acceleration and angular acceleration information [α t t , w t b ] of the UAV with the IMU, and the coordinate system of the information is UAV coordinate system; based on the state propagation model of the IMU, the position, speed and angle information of the UAV can be obtained at every moment as follows:

{

u
t

=

[

a
t
t

,

w
t
b

]

T

v
t

=

[
a

v
t

,

a

v
t

]

T

[

P

t
+
1

w

,

P
.

t
+
1

b

,

∅

t
+
1

w

]

=

f

Assignees

• Univ Tongji

Inventors

• He Bin
• Li Gang
• Shen Runjie
• Zhu Zhongpan
• Wang Zhipeng
• Chen Jie
• Wang Xudong

Classifications

• G05D1/46

  Control of position or course in three dimensions [3D] · CPC title

• G05D2109/20

  Aircraft, e.g. drones · CPC title

• G06T5/92

  based on global image properties · CPC title

• G06T5/70

  Denoising; Smoothing · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title