Structure from motion (SfM) processing for unmanned aerial vehicle (UAV)

Patent metadata

Field	Value
Publication number	US-11086324-B2
Application number	US-201816201335-A
Country	US
Kind code	B2
Filing date	Nov 27, 2018
Priority date	Jul 21, 2014
Publication date	Aug 10, 2021
Grant date	Aug 10, 2021

Abstract

A method of imaging an area using an unmanned aerial vehicle (UAV) collects a plurality of images from a sensor mounted to the UAV. The plurality of images are processed to detect regions that require additional imaging and an updated flight plan and sensor gimbal position plan is created to capture portions of the area identified as requiring additional imaging.

First claim

The invention claimed is:

1. A method of collecting images from an unmanned aerial vehicle (UAV), the method comprising:
capturing images utilizing a camera positioned on a camera gimbal assembly associated with the UAV;
receiving UAV positional information from a flight controller that indicates a current position of the UAV;
receiving UAV orientation information from an inertial measurement unit (IMU) regarding current orientation of the UAV;
receiving orientation information from the camera gimbal assembly located on the UAV regarding current gimbal assembly position and/or orientation of the camera gimbal assembly;
calculating in real-time a 3D coordinate associated with captured images based on the received UAV positional information, the received UAV orientation information, and the received gimbal assembly position and/or orientation information, wherein the 3D coordinates associated with each captured image are utilized to detect overlap between captured images, wherein captured images that do not overlap with other captured images are discarded; and
determining, in-flight, the sufficiency of captured image data.

2. The method of claim 1 , wherein the 3D coordinate is paired with the captured images at a time of capture and associated with the captured image data.

3. The method of claim 1 , wherein the received UAV positional information, the received UAV orientation information, and the received gimbal assembly position and/or orientation information is associated with the captured image data and communicated to a ground-based system, wherein the ground-based system calculates the 3D coordinate to be associated with captured imaged captured.

4. The method of claim 1 , wherein point-cloud reconstructions of scanned structures are created utilizing the captured image data, wherein density of points within a given region is utilized to determine sufficiency of captured image data.

5. The method of claim 1 , wherein overlapping captured images are organized into groups, wherein groups of captured images are processed separately.

6. The method of claim 5 , wherein point-cloud reconstruction of each group is utilized to determine the sufficiency of captured image data.

Assignees

• Univ King Abdullah Sci & Tech

Inventors

• Smith Neil G
• Shalaby Mohamed
• Passone Luca

Classifications

• G01C21/1656

  with passive imaging devices, e.g. cameras · CPC title

• B64U2201/104

  using satellite radio beacon positioning systems, e.g. GPS · CPC title

• B64U2101/32

  for cartography or topography · CPC title

• B64U30/20

  Rotors; Rotor supports · CPC title

• B64U10/14

  with four distinct rotor axes, e.g. quadcopters · CPC title