Aircraft positioning on a taxiway

Patent metadata

Field	Value
Publication number	US-11024187-B2
Application number	US-201816224885-A
Country	US
Kind code	B2
Filing date	Dec 19, 2018
Priority date	Dec 19, 2018
Publication date	Jun 1, 2021
Grant date	Jun 1, 2021

Abstract

Systems, methods, and computer-readable media storing instructions for determining cross-track error of an aircraft on a taxiway are disclosed herein. The disclosed techniques capture electronic images of a portion of the taxiway using cameras or other electronic imaging devices mounted on the aircraft, pre-process the electronic images to generate regularized image data, apply a trained multichannel neural network model to the regularized image data to generate a preliminary estimate of cross-track error relative to the centerline of the taxiway, and post-process the preliminary estimate to generate an estimate of cross-track error of the aircraft. Further embodiments adjust a GPS-based location estimate of the aircraft using the estimate of cross-track error or adjust the heading of the aircraft based upon the estimate of cross-track error.

First claim

What is claimed:

1. An aircraft guidance or control system for an aircraft, comprising:
one or more processors;
a program memory storing (i) a multichannel neural network model and (ii) executable instructions that, when executed by the one or more processors, cause the aircraft guidance or control system to:
receive a plurality of electronic images from a plurality of electronic imaging devices mounted on the aircraft, wherein the plurality of electronic imaging devices are mounted to capture portions of a taxiway while the aircraft is sitting on or traveling along the taxiway;
process the plurality of electronic images to generate regularized image data;
based upon the regularized image data, generate a preliminary estimate of a cross-track error of the aircraft relative to a centerline position of the taxiway by applying the multichannel neural network model to the regularized image data; and
post-process the preliminary estimate of the cross-track error to generate an estimate of the cross-track error using one or more previous estimates of one or more previous cross-track errors of the aircraft.

2. The aircraft guidance or control system of claim 1 , wherein the executable instructions further cause the aircraft guidance or control system to:
receive a location estimate of the aircraft from a Global Positioning System (GPS) unit of the aircraft; and
adjust the location estimate based upon the estimate of the cross-track error.

3. The aircraft guidance or control system of claim 1 , wherein the executable instructions further cause the aircraft guidance or control system to:
determine an adjustment to a heading of the aircraft to reduce the cross-track error by directing the aircraft toward the centerline position of the taxiway; and
adjust a rudder control of the aircraft to implement the adjustment to the heading of the aircraft.

4. The aircraft guidance or control system of claim 1 , wherein the executable instructions that cause the aircraft guidance or control system to process the plurality of electronic images cause the aircraft guidance or control system to:
remove one of more portions of each of the plurality of electronic images, corresponding to one or more of the following: sky, a propeller of the aircraft, a wing of the aircraft, or a body of the aircraft; and
resize each of the plurality of electronic images to a standard size.

5. The aircraft guidance or control system of claim 1 , wherein the executable instructions that cause the aircraft guidance or control system to post-process the preliminary estimate of the cross-track error cause the aircraft guidance or control system to:
apply a Kalman filter to the preliminary estimate of the cross-track error to smooth changes to estimates of cross-track error over time using the one or more previous estimates of the one or more previous cross-track errors.

6. The aircraft guidance or control system of claim 1 , wherein the executable instructions further cause the aircraft guidance or control system to:
calibrate the plurality of electronic imaging devices based upon one or more positions of one or more external portions of the aircraft within the plurality of electronic images.

7. The aircraft guidance or control system of claim 1 , further comprising the plurality of electronic imaging devices, wherein:
the plurality of electronic imaging devices includes at least a left wing electronic imaging device mounted on a left wing of the aircraft and a right wing electronic imaging device mounted on a right wing of the aircraft;
the plurality of electronic images includes a left channel having electronic images from the left wing electronic imaging device and a right channel having electronic images from the right wing electronic imaging device; and
the multichannel neural network model is configured to receive pre-processed versions of the left channel and the right channel in the regularized image data.

8. A computer-implemented method for aircraft guidance or control, comprising:
accessing, by one or more processors, a multichannel neural network model stored in a program memory;
receiving, at one or more processors, a plurality of electronic images from a plurality of electronic imaging devices mounted on an aircraft, wherein the plurality of electronic imaging devices are mounted to capture portions of a taxiway while the aircraft is sitting on or traveling along the taxiway;
processing, by the one or more processors, the plurality of electronic images to generate regularized image data;
generating, by the one or more processors and based upon the regularized image data, a preliminary estimate of a cross-track error of the aircraft relative to a centerline position of the taxiway by applying the multichannel neural network model to the regularized image data; and
post-processing, by the one or more processors, the preliminary estimate of the cross-track error to generate an estimate of the cross-track error using one or more previous estimates of one or more previous cross-track errors of the aircraft.

9. The computer-implemented method of claim 8 , further comprising:
receiving, at the one or more processors, a location estimate of the aircraft from a Global Positioning System (GPS) unit of the aircraft; and
adjusting, by the one or more processors, the location estimate based upon the estimate of the cross-track error.

10. The computer-implemented method of claim 8 , further comprising:
determining, by the one or more processors, an adjustment to a heading of the aircraft to reduce the cross-track error by directing the aircraft toward the centerline position of the taxiway; and
adjusting, by the one or more processors, a rudder control of the aircraft to implement the adjustment to the heading of the aircraft.

11. The computer-implemented method of claim 8 , wherein:
processing the plurality of electronic images includes removing one of more portions of each of the plurality of electronic images, corresponding to one or more of the following: sky, a propeller of the aircraft, a wing of the aircraft, or a body of the aircraft; and
post-processing the preliminary estimate of the cross-track error includes applying a Kalman filter to the preliminary estimate of the cross-track error to smooth changes to estimates of cross-track error over time using the one or more previous estimates of the one or more previous cross-track errors.

12. The computer-implemented method of claim 8 , further comprising:
generating, by one or more measurement devices and one or more training electronic imaging devices, a training dataset containing a plurality of data points, each data point including the following: (i) a set of a plurality of training images and (ii) an error measurement associated with the set of training images and indicating a distance from a training centerline of a training taxiway;
generating, by one or more additional processors, the multichannel neural network model by:
accessing the training dataset containing the plurality of data points; and
training a base model by applying a training algorithm to the data points in the training dataset as inputs to obtain the multichannel neural network model; and

storing, in the program memory, the multichannel neural network model.

13. The computer-implemented method of claim 12 , wherein generating the multichannel neural network model further comprises:
generating augmented data points by applying one or more of the following data augmentations to one or more of the training images of one or more of the data points in the training dataset: vertical jitter, rotation, or anomalous image artifacts; and

Assignees

• Boeing Co

Inventors

• Staudinger Tyler
• Callahan Kevin S
• Chang Isaac
• Dame Stephen
• Evans Nick
• Jorgensen Zachary
• Kalin Joshua
• Muir Eric

Classifications

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G08G5/21

  located onboard the aircraft · CPC title

• G08G5/50 Primary

  Navigation or guidance aids · CPC title