Cross spectral feature correlation for navigational adjustment

Patent metadata

Field	Value
Publication number	US-2016350927-A1
Application number	US-201514725964-A
Country	US
Kind code	A1
Filing date	May 29, 2015
Priority date	May 29, 2015
Publication date	Dec 1, 2016
Grant date	—

Abstract

A system includes a sensor to generate a first image having a first two-dimensional image pixel data set. A database provides a second image having a second two-dimensional image pixel data set that includes a three-dimensional positional data set describing a navigational position of each pixel in the second two-dimensional image pixel data set. A vision module includes an edge extractor to extract image edge features from the first two-dimensional pixel data set and image edge features from the second two-dimensional image pixel data set. The vision module includes a feature correlator to determine a navigational position for each pixel in the first two-dimensional data set based on an image edge feature comparison of the extracted edge features from the first and second two-dimensional image pixel data sets.

First claim

What is claimed is:

1 . A system, comprising:
a sensor to generate a first image having a first two-dimensional image pixel data set; a database to provide a second image having a second two-dimensional image pixel data set that includes a three-dimensional positional data set describing a navigational position of each pixel in the second two-dimensional image pixel data set; and a vision module comprising:
an edge extractor to extract image edge features from the first two-dimensional pixel data set and image edge features from the second two-dimensional image pixel data set; and
a feature correlator to determine a navigational position for each pixel in the first two-dimensional data set based on an image edge feature comparison of the extracted edge features from the first and second two-dimensional image pixel data sets, wherein the feature correlator assigns a subset of the three-dimensional positional data set from the second image to a subset of correlated features in the first image based on the image edge feature comparison.

2 . The system of claim 1 , wherein the first image is captured from an infrared frequency sensor and the second image is stored in the database from a visible frequency sensor.

3 . The system of claim 1 , wherein the edge extractor performs a gradient operation and identifies edge point pixel positions of the first and second image by selecting gradient magnitudes above a predetermined threshold to identify the edge point pixel positions.

4 . The system of claim 3 , wherein the gradient operation includes applying a gradient operator that includes at least one of a Roberts operator, a Prewitt operator, a Sobel operator, and an Isotropic operator.

5 . The system of claim 1 , further comprising a filter to transform extracted edge features from the first and second image pixel data sets into a first smooth image domain and a second smooth image domain.

6 . The system of claim 5 , wherein the filter performs a Gaussian filter to transform the extracted edge features from the first and second image pixel data sets into the first smooth image domain and a second smooth image domain.

7 . The system of claim 6 , further comprising an orientation calculator that employs a histogram to determine an orientation between edge features of the first smooth image domain and the second smooth image domain.

8 . The system of claim 7 , wherein the feature correlator computes a distance between the histogram of the oriented edge features of the first and second smooth image domains for a subset of pixels in the first image pixel data set based on the computed distance to a subset of the second image pixel data set.

9 . The system of claim 8 , wherein the feature correlator ranks the computed distance between the oriented edge features of the first and second smooth image domains and determines a navigational position for a subset of pixels in the first image pixel data set based on the ranking in order to extract the three-dimensional position data associated with the second image pixel data set.

10 . The system of claim 9 , further comprising a navigational system that receives the navigational position to correct a navigation solution for the navigational system.

11 . The system of claim 1 , further comprising a warping module to change the angular prospective of the first image from about a 45 degree prospective view to a top-down view of the first image.

12 . A method, comprising:
transforming a first image pixel data set, via a processor, to a first image edge domain, transforming a second image pixel data set, via the processor, to a second image edge domain, wherein each pixel in the second image pixel data set is associated with three-dimensional position data; filtering the first and second image edge domains, via the processor, to transform the first and second image edge domains into a first smooth image domain and a second smooth image domain; calculating, via the processor, a histogram for each of the first and second smooth image domains to determine an orientation between the first and second smooth image domains; and computing a distance, via the processor, between the histograms of the oriented edge features of the first and second smooth image domains to determine a navigational position for a subset of pixels in the first image pixel data set based on a correlation to the second image pixel data set in order to extract the three-dimensional position data associated with the second image pixel data set.

13 . The method of claim 12 , further comprising performing a gradient operation and identifying edge point pixel positions of the first and second image by selecting gradient magnitudes above a predetermined threshold to identify the edge point pixel positions.

14 . The method of claim 13 , wherein the gradient operation includes applying a gradient operator that includes at least one of a Roberts operator, a Prewitt operator, a Sobel operator, and an Isotropic operator.

15 . The method of claim 12 , further comprising performing a Gaussian filter to transform the extracted edge features from the first and second image pixel data sets into the first smooth image domain and a second smooth image domain.

16 . The method of claim 12 , further comprising ranking the computed distance between the histogram of the oriented edge features of the first and second smooth image domains and determining a navigational position for a subset of pixels in the first image pixel data set based on the ranking in order to extract the three-dimensional position data associated with the second image pixel data set.

17 . The method of claim 16 , further comprising updating a navigational system with the navigational position to correct a navigation solution for the navigational system.

18 . A non-transitory computer readable medium having computer executable instructions stored thereon, the computer executable instructions configured to:
transform an infrared image pixel data set from a sensor to a first image edge domain, transform a visual image pixel data set from a database to a second image edge domain, wherein each pixel in the visual image pixel data set is associated with three-dimensional position data; filter the first and second image edge domains to transform the first and second image edge domains into a first smooth image domain and a second smooth image domain; calculate a histogram for each of the first and second smooth image domains to determine an orientation between the first and second smooth image domains; and compute a distance between the histograms of the oriented edge features of the first and second smooth image domains to determine a navigational position for a subset of pixels in the infrared image pixel data set based on a correlation to the visual image pixel data set in order to extract the three-dimensional position data associated with the visual image pixel data set.

19 . The non-transitory computer readable medium of claim 18 , further comprising computer executable instructions to perform a gradient operation and identify edge point pixel positions of the first and second image by selecting gradient magnitudes above a predetermined threshold to identify the edge point pixel positions.

20 . The non-transitory computer readable medium of claim 19 , further comprising computer executable instructions to perform a Gaussian filter to transform the first and second image edge domains into a first smooth image domain and a second smooth

Assignees

• Ma Yunqian
• Northrop Grumman Systems Corp

Inventors

• Ma Yunqian

Classifications

• G06T2207/20072

  Graph-based image processing · CPC title

• G06T2207/20182

  Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering · CPC title

• G06T7/74 Primary

  involving reference images or patches · CPC title

• G06T2207/20024

  Filtering details · CPC title

• G06T5/20

  using local operators · CPC title