System, method, and computer readable medium for autonomous airport runway navigation

Patent metadata

Field	Value
Publication number	US-10878709-B2
Application number	US-201816039490-A
Country	US
Kind code	B2
Filing date	Jul 19, 2018
Priority date	Jul 19, 2018
Publication date	Dec 29, 2020
Grant date	Dec 29, 2020

Abstract

Example implementations relate to autonomous airport runway navigation. An example system includes a first sensor and a second sensor coupled to an aircraft at a first location and a second location, respectively, and a computing system configured to receive sensor data from one or both of the first sensor and the second sensor to detect airport markings positioned proximate a runway. The computing system is further configured to identify a centerline of the runway based on the airport markings and receive sensor data from both of the first sensor and the second sensor to determine a lateral displacement that represents a distance between a reference point of the aircraft and the centerline of the runway. The computing system is further configured to control instructions that indicate adjustments for aligning the reference point of the aircraft with the centerline of the runway during subsequent navigation of the aircraft.

First claim

What is claimed is:

1. A system, comprising:
a first sensor coupled to an aircraft at a first location;
a second sensor coupled to the aircraft at a second location;
an attitude and heading reference system (AHRS); and
a computing system configured to:
receive a first set of sensor data that includes sensor data from one or both of the first sensor and the second sensor;
detect one or more airport markings positioned proximate a runway using the first set of sensor data, wherein the aircraft is positioned on the runway;
identify a centerline of the runway based on the one or more airport markings;
receive a second set of sensor data that includes sensor data from both of the first sensor and the second sensor and trend information from the AHRS;
determine a lateral displacement that represents a distance between a reference point of the aircraft and the centerline of the runway using the second set of sensor data, wherein the lateral displacement is initially determined based on a comparison of a first position of the reference point relative to the centerline of the runway as represented in sensor data from the first sensor with a second position of the reference point relative to the centerline of the runway as represented in sensor data from the second sensor and refined based on a future position and orientation of the aircraft determined using the trend information; and
provide control instructions that indicate one or more adjustments for aligning the reference point of the aircraft with the centerline of the runway during subsequent navigation of the aircraft.

2. The system of claim 1 , wherein the computing system is further configured:
detect noise in the second set of sensor data received from both of the first sensor and the second sensor; and
responsive to detecting the noise in the second set of sensor data, use only the future position and orientation of the aircraft to determine the lateral displacement that represents the distance between the reference point of the aircraft and the centerline of the runway.

3. The system of claim 1 , wherein the computing system is further configured:
detect a gap in the centerline of the runway using the second set of sensor data received from both of the first sensor and the second sensor; and
responsive to detecting the gap in the centerline of the runway, use only the future position and orientation of the aircraft to determine the lateral displacement that represents the distance between the reference point of the aircraft and the centerline of the runway.

4. The system of claim 1 , wherein the first location is on a first wing of the aircraft such that the first sensor receives sensor data of an area that includes the reference point of the aircraft and corresponding runway positioned below the reference point of the aircraft at a first perspective, and wherein the second location is on a second wing of the aircraft such that the second sensor receives sensor data of the area that includes the reference point of the aircraft and corresponding runway positioned below the reference point of the aircraft at a second perspective.

5. The system of claim 1 , wherein the first sensor and the second sensor include cameras.

6. The system of claim 1 , wherein the reference point is a nose of the aircraft.

7. The system of claim 1 , wherein the computing system is configured to provide the control instructions that indicate the one or more adjustments for aligning the reference point of the aircraft with the centerline of the runway during subsequent navigation of the aircraft to a remote user interface, wherein the remote user interface is configured to display the one or more adjustments and enable remote control of the aircraft.

8. The system of claim 7 , wherein the remote user interface includes a head-mountable display.

9. The system of claim 1 , wherein the computing system is configured to provide the control instructions that indicate the one or more adjustments for aligning the reference point of the aircraft with the centerline of the runway during subsequent navigation of the aircraft to a control system of the aircraft for autonomous operation.

10. The system of claim 1 , wherein the computing system is further configured to:
identify an intersection of the runway based on the one or more airport markings, wherein an airport marking corresponds to a surface line of the runway or a sign positioned proximate the runway; and
wherein the computing system is configured to identify the centerline of the runway based in part on the intersection.

11. The system of claim 10 , wherein the computing system is further configured to:
identify a boundary of the runway based on the one or more airport markings; and
wherein the computing system is configured to identify the centerline of the runway based in part on the boundary of the runway.

12. A method comprising:
receiving, at a computing system, a first set of sensor data, wherein the first set of sensor data includes sensor data from one or both of a first sensor coupled to an aircraft at a first location and a second sensor coupled to the aircraft at a second location;
detecting, by the computing system, one or more airport markings positioned proximate a runway using the first set of sensor data, wherein the aircraft is positioned on the runway;
identifying a centerline of the runway based on the one or more airport markings;
receiving, at the computing system, a second set of sensor data, wherein the second set of sensor data includes sensor data from both of the first sensor and the second sensor and trend information from an attitude and heading reference system (AHRS);
determining, by the computing system, a lateral displacement that represents a distance between a reference point of the aircraft and the centerline of the runway using the second set of sensor data, wherein the lateral displacement is initially determined based on a comparison of a first position of the reference point of the aircraft relative to the centerline of the runway as represented in sensor data from the first sensor with a second position of the reference point of the aircraft relative to the centerline of the runway as represented in sensor data from the second sensor and refined based on a future position and orientation of the aircraft determined using the trend information; and
providing, by the computing system, control instructions that indicate one or more adjustments for aligning the reference point of the aircraft with the centerline of the runway during subsequent navigation of the aircraft.

13. The method of claim 12 , further comprising:
detecting noise in the second set of sensor data received from both of the first sensor and the second sensor; and
based on detecting noise in the second set of sensor data, determining the lateral displacement that represents the distance between the reference point of the aircraft and the centerline of the runway only using the future position and orientation of the aircraft.

14. The method of claim 12 , further comprising:
detecting a gap in the centerline of the runway using the second set of sensor data received from both of the first sensor and the second sensor; and
based on detecting the gap in the centerline of the runway, determining the lateral displacement that represents the distance between the reference point of the aircraft and the centerline of the runway only using the future position and orientation of the aircraft.

15. The method of claim 12 , wherein providing control instructions that indicate one or more adjustments for aligning the refe

Assignees

• Boeing Co

Inventors

• Dame Stephen
• Margineantu Dragos D
• Evans Nick S
• Staudinger Tyler C
• Rupnik Brian K
• Moser Matthew A
• Callahan Kevin S
• Whitehead Brian T

Classifications

• G08G5/51 Primary

  for control when on the ground, e.g. taxiing or rolling · CPC title

• G05D1/0202

  specially adapted to aircraft · CPC title

• G05D1/0083

  to help an aircraft pilot in the rolling phase · CPC title

• G05D1/0038

  by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation (images analyzed by a computer and used for automatic navigation G05D1/0246) · CPC title

• G08G5/21 Primary

  located onboard the aircraft · CPC title