Aircraft door camera system for wing monitoring

Patent metadata

Field	Value
Publication number	US-12100307-B2
Application number	US-202217886181-A
Country	US
Kind code	B2
Filing date	Aug 11, 2022
Priority date	Aug 11, 2021
Publication date	Sep 24, 2024
Grant date	Sep 24, 2024

Abstract

A camera with a field of view toward an external environment of an aircraft is disposed within an aircraft door such that a leading edge of a wing of the aircraft is within the field of view of the camera. A display device is disposed within an interior of the aircraft. A processor is operatively coupled to the camera and to the display device. The processor analyzes image data captured by the camera to predict a likelihood of foreign object collision with the leading edge of the wing, or detect damage or deformation to the leading edge.

First claim

The invention claimed is:

1. A system for monitoring an external environment of an aircraft, the system comprising:
an aircraft door;
a camera with a field of view toward the external environment of the aircraft and disposed within the aircraft door such that a leading edge of a wing of the aircraft is within the field of view of the camera;
a display device disposed within an interior of the aircraft; and
a processor operatively coupled to the camera and to the display device to:
receive image data captured by the camera that is representative of the external environment of the aircraft;
output the captured image data for display at the display device; and
analyze the image data by:
identifying a region within the captured image data that corresponds to the leading edge of the wing;
monitoring the leading edge of the wing, based on the captured image data corresponding to the leading edge of the wing;
accessing pixel coordinates of captured image data associated with the leading edge of the wing in a baseline state;
causing a computer-readable memory to store the pixel coordinates corresponding to the leading edge of the wing in the baseline state;
comparing the pixel coordinates corresponding to the leading edge of the wing during operation with the pixel coordinates corresponding to the leading edge of the wing in the baseline state; and
producing a warning associated with the leading edge of the wing in response to the captured image data indicating anticipated future collision to the leading edge of the wing, damage to the leading edge of the wing, or change to the leading edge of the wing.

2. The system of claim 1 , wherein the processor is operatively coupled to the camera and to the display device to analyze the image data for an anticipated future collision by:
determining a motion vector of the region that corresponds to the leading edge of the wing within the captured image data;
identifying a region within the captured image data that corresponds to an object separate from the aircraft;
determining a motion vector of the region that corresponds to the object separate from the aircraft within the captured image data;
determining, based on the motion vector of the region corresponding to the leading edge of the wing and the motion vector corresponding to the object separate from the aircraft, an anticipated future collision location within the captured image data; and
producing a warning of anticipated collision based on the anticipated future collision location.

3. The system of claim 2 , wherein the processor is operatively coupled to the camera to analyze the image data to determine the motion vector of the region corresponding to the leading edge of the wing using multiple frames of captured image data.

4. The system of claim 3 , wherein the processor is operatively coupled to the camera to utilize an optical flow algorithm to determine the motion vector of the region corresponding to the leading edge of the wing and the motion vector corresponding to the object separate from the aircraft.

5. The system of claim 2 , wherein the processor is operatively coupled to the camera to identify a region within the captured image data that corresponds to an object separate from the aircraft by utilizing an object detection algorithm.

6. The system of claim 5 , wherein the object detection algorithm is a real-time neural network-based algorithm trained using baseline image data of objects.

7. The system of claim 1 , wherein the processor is operatively coupled to the camera and to the display device to analyze the image data for damage to the leading edge of the wing or change to the leading edge of the wing by:
determining whether the region within the captured image data that corresponds to the leading edge of the wing conforms to baseline image data corresponding to the leading edge of the wing; and
producing a warning associated with the leading edge of the wing in response to determining that the region within the image data that corresponds to the leading edge of the wing does not conform to the baseline image data indicating damage to the leading edge of the wing or change to the leading edge of the wing.

8. The system of claim 7 , wherein the processor is operatively coupled to the camera to:
generate a first vector of pixel coordinates associated with the leading edge of the wing during operation;
generate a second vector of pixel coordinates associated with the leading edge of the wing in the baseline state;
determine an angle between the first vector of pixel coordinates and the second vector of pixel coordinates which represents an extent of deviation between the leading edge of the wing during operation and the leading edge of the wing in the baseline state; and
determine whether the angle between the first vector of pixel coordinates and the second vector of pixel coordinates exceeds a threshold angle indicating damage to the leading edge of the wing or change to the leading edge of the wing.

9. The system of claim 1 , wherein the processor is operatively coupled to the camera to utilize an edge detection algorithm to identify the leading edge of the wing.

10. The system of claim 9 , wherein the processor is operatively coupled to the camera to utilize a Canny edge detector multi-stage algorithm to identify strong edges within the captured image data.

11. The system of claim 1 , wherein the processor is operatively coupled to the camera to categorize a region of pixels about the leading edge of the wing identified in the captured image data as corresponding to the leading edge of the wing.

12. The system of claim 1 , wherein the warning is a visual alert for display at the display device, an audible alarm, or an alert notification.

13. A method of monitoring an external environment of an aircraft, the method comprising:
receiving, with a processor, image data captured by a camera disposed within an aircraft door of the aircraft such that a leading edge of a wing of the aircraft is within a field of view of the camera;
analyzing the captured image data by:
identifying, with the processor, a region within the captured image data that corresponds to the leading edge of the wing;
monitoring, with the processor, the leading edge of the wing based on the captured image data corresponding to the leading edge of the wing;
accessing pixel coordinates of captured image data associated with the leading edge of the wing in a baseline state;
causing a computer-readable memory to store the pixel coordinates corresponding to the leading edge of the wing in the baseline state;
comparing the pixel coordinates corresponding to the leading edge of the wing during operation with the pixel coordinates corresponding to the leading edge of the wing in the baseline state;

producing a warning associated with the leading edge of the wing based upon the captured image data indicating anticipated future collision to the leading edge of the wing, damage to the leading edge of the wing, or change to the leading edge of the wing; and

outputting the captured image data for display at a display device disposed within an interior of the aircraft.

14. The method of claim 13 , further comprising:
determining a motion vector of the region that corresponds to the leading edge of the wing within the captured image data;
identifying a region within the captured image data that corresponds to an object separate from the aircraft;
determining a motion vector of the region that corresponds to the object separate from the aircraft within the captured image data; and
determining, based on the motion vector of the r

Assignees

• Rosemount Aerospace Inc

Inventors

• Pesik Joseph T
• Boer Jonathan

Classifications

• G08G5/51 Primary

  for control when on the ground, e.g. taxiing or rolling · CPC title

• G06T2207/30241

  Trajectory · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06T2207/20076

  Probabilistic image processing · CPC title

• G06V10/759

  Region-based matching · CPC title