Vision-aided aerial navigation

Patent metadata

Field	Value
Publication number	US-10054445-B2
Application number	US-201615155818-A
Country	US
Kind code	B2
Filing date	May 16, 2016
Priority date	May 16, 2016
Publication date	Aug 21, 2018
Grant date	Aug 21, 2018

Abstract

An aerial vehicle is navigated using vision-aided navigation that classifies regions of acquired still image frames as featureless or feature-rich, and thereby avoids expending time and computational resources attempting to extract and match false features from the featureless regions. The classification may be performed by computing a texture metric as by testing widths of peaks of the autocorrelation function of a region against a threshold, which may be an adaptive threshold, or by using a model that has been trained using a machine learning method applied to a training dataset comprising training images of featureless regions and feature-rich regions. Such machine learning method can use a support vector machine. The resultant matched feature observations can be data-fused with other sensor data to correct a navigation solution based on GPS and/or IMU data.

First claim

I claim:

1. A method of navigation of an aerial vehicle, the method comprising:
acquiring an aerial image frame from a camera on the aerial vehicle;
partitioning the acquired aerial image frame into a plurality of regions;
classifying one or more of the plurality of regions as featureless or feature-rich by computing an autocorrelation function to determine at least one peak width, and testing the at least one peak width against a threshold;
extracting features for those regions classified as feature-rich and not for those regions classified as featureless; and
navigating the aerial vehicle based at least in part on the extracted features.

2. The method of claim 1 , wherein the classifying of the regions is conducted in parallel, with each region classified in an independent software thread.

3. The method of claim 1 , further comprising matching features following the extracting features and prior to the navigating the aerial vehicle, wherein matched feature observations are generated by matching features extracted from the regions in the feature extraction to corresponding features in a map feature database.

4. The method of claim 1 , further comprising performing perspective warping following the classifying and prior to the extracting features, wherein the perspective of a region is ortho-rectified.

5. The method of claim 1 , wherein the partitioning the acquired aerial image frame comprises partitioning the acquired aerial image frame into regions using a watershed segmentation method.

6. The method of claim 1 , wherein the partitioning the acquired aerial image frame comprises partitioning the acquired aerial image frame into regions using a grid partition method.

7. The method of claim 3 , further comprising performing data fusion following the matching features and prior to the navigating the aerial vehicle, wherein the matched feature observations are combined with inertial measurement unit (IMU) sensor readings using a Kalman filter to arrive at a corrected navigation solution suitable for input into an inertial navigation system (INS).

8. The method of claim 7 , wherein the Kalman filter fuses readings from a barometric sensor in addition to the matched feature observations and the IMU sensor readings to arrive at the corrected navigation solution.

9. The method of claim 1 , wherein the threshold is dynamically computed as an Otsu threshold statistically derived from a set of peak widths determined from the plurality of regions, the set of peak widths having a bimodal distribution.

10. A system for navigation of an aerial vehicle comprising:
a camera configured to acquire an aerial image frame;
an image processor comprising:
a partitioner configured to partition an acquired aerial image frame into a plurality of regions;
a classifier configured to classify one or more of the plurality of regions as featureless or feature-rich by computing an autocorrelation function to determine at least one peak width, and testing the at least one peak width against a threshold; and
a feature extractor configured to extract features only for those regions classified as feature-rich and not for those regions classified as featureless; and

navigation controls configured to navigate the aerial vehicle based at least in part on the extracted features.

11. The system of claim 10 , wherein the classifier is configured to classify the regions in parallel, with each region classified in an independent software thread.

12. The system of claim 10 , further comprising a feature matcher configured to generate matched feature observations by matching features extracted from the regions by the feature extractor to corresponding features in a map feature database.

13. The system of claim 12 , further comprising a data fusion engine configured to generate a corrected navigation solution by combining the matched feature observations with inertial measurement unit (IMU) and global positioning system (GPS) sensor readings using a Kalman filter, the corrected navigation solution suitable for input into an inertial navigation system (INS).

14. The system of claim 13 , wherein the Kalman filter fuses readings from a barometric sensor in addition to the matched feature observations and the IMU and GPS sensor readings to arrive at the corrected navigation solution.

15. The system of claim 10 , wherein the classifier dynamically computes the threshold as an Otsu threshold statistically derived from a set of peak widths determined from the plurality of regions, the set of peak widths having a bimodal distribution.

16. A method of navigation of an aerial vehicle comprising:
acquiring an aerial image frame from a camera on the aerial vehicle;
partitioning the acquired aerial image frame into regions;
classifying one or more regions as only one of featureless or feature-rich, before extracting features from said regions, the classifying using a model that has been trained using a machine learning method applied to a training dataset comprising a plurality of training images of featureless regions and a plurality of training images of feature-rich regions;
extracting, from the aerial image frame, features for those regions classified as feature-rich and not for those regions classified as featureless; and
navigating the aerial vehicle based at least in part on the extracted features.

17. The method of claim 16 , wherein the machine learning method uses a support vector machine.

18. The method of claim 16 , wherein the model has been trained using a machine-learning method applied to features extracted from the training images prior to acquiring the aerial image frame.

19. The method of claim 16 , further comprising matching features following the extracting features and prior to the navigating the aerial vehicle, wherein matched feature observations are generated by matching features extracted from the regions in the feature extraction step to corresponding features in a map feature database.

20. The method of claim 19 , further comprising performing data fusion following the matching features and prior to the navigating the aerial vehicle, wherein the matched feature observations are combined with inertial measurement unit (IMU) sensor readings using a Kalman filter to arrive at a corrected navigation solution suitable for input into an inertial navigation system (INS).

Assignees

• Ma Yunqian
• Northrop Grumman Systems Corp

Inventors

• Ma Yunqian

Classifications

• G01C21/005

  with correlation of navigation data from several sources, e.g. map or contour matching (G01C21/30 takes precedence) · CPC title

• G06V20/17

  taken from planes or by drones · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• B64U2201/20

  Remote controls · CPC title