Methods and systems for acoustic machine perception for an aircraft

Patent metadata

Field	Value
Publication number	US-12061256-B2
Application number	US-202217988001-A
Country	US
Kind code	B2
Filing date	Nov 16, 2022
Priority date	Jun 13, 2019
Publication date	Aug 13, 2024
Grant date	Aug 13, 2024

Abstract

In an example, a method is described. The method includes causing one or more sensors arranged on an aircraft to acquire, over a window of time, first data associated with a first object that is within an environment of the aircraft, where the one or more sensors include one or more of a light detection and ranging (LIDAR) sensor, a radar sensor, or a camera, causing an array of microphones arranged on the aircraft to acquire, over approximately the same window of time as the first data is acquired, first acoustic data associated with the first object, and training a machine learning model by using the first acoustic data as an input value to the machine learning model and by using an azimuth, a range, an elevation, and a type of the first object identified from the first data as ground truth output labels for the machine learning model.

First claim

What is claimed is:

1. A method comprising:
causing one or more sensors arranged on an aircraft to acquire, over a window of time, first data associated with a first object that is within an environment of the aircraft while the aircraft is in motion, wherein the one or more sensors include one or more of a light detection and ranging sensor, a radar sensor, or a camera;
causing an array of microphones arranged on the aircraft to acquire, over approximately the same window of time as the first data is acquired, first acoustic data associated with the first object while the aircraft is in motion; and
training, by a processor, a machine learning model by using the first acoustic data as an input value to the machine learning model and by using one or more of an azimuth of the first object, a range of the first object, an elevation of the first object, or a type of the first object identified from the first data as ground truth output labels for the machine learning model,
wherein the machine learning model is configured to predict, based on target acoustic data subsequently acquired by the array of microphones and associated with a target object within the environment of the aircraft, one or more of an azimuth of the target object, a range of the target object, an elevation of the target object, or a type of the target object.

2. The method of claim 1 , wherein causing the array of microphones arranged on the aircraft to acquire the first acoustic data associated with the first object comprises causing one or more microphones arranged on each wing of the aircraft to acquire the first acoustic data associated with the first object.

3. The method of claim 1 , wherein the azimuth of the first object, the range of the first object, and the elevation of the first object identified from the first data are relative to the aircraft, the method further comprising:
detecting, by the processor, over approximately the same window of time as the first data and the first acoustic data are acquired, automatic dependent surveillance-broadcast information broadcast by the first object,
wherein the automatic dependent surveillance-broadcast information indicates one or more of an azimuth of the first object relative to the Earth, a range of the first object relative to the Earth, an elevation of the first object relative to the Earth, the type of the first object, a heading of the first object, or a speed of the first object, and
wherein training the machine learning model comprises training the machine learning model by using one or more of the azimuth of the first object relative to the Earth, the range of the first object relative to the Earth, the elevation of the first object relative to the Earth, the type of the first object, the heading of the first object, or the speed of the first object indicated by the automatic dependent surveillance-broadcast information broadcast by the first object as at least a portion of the ground truth output labels for the machine learning model.

4. The method of claim 1 , further comprising:
receiving, by the processor, over approximately the same window of time as the first data and the first acoustic data are acquired, information indicating a state of an engine of the aircraft,
wherein training the machine learning model comprises training the machine learning model by using the state of the engine as one of the ground truth output labels for the machine learning model,
wherein the machine learning model is configured to predict, based on the target acoustic data, that the engine is generating an anomalous acoustic signature.

5. The method of claim 1 , further comprising:
determining, over approximately the same window of time as the first data and the first acoustic data are acquired, a global positioning system location of the aircraft;
receiving a weather report; and
determining one or more weather conditions of the environment indicated by the weather report as being present at the global positioning system location over approximately the same window of time as the first data and the first acoustic data are acquired,
wherein training the machine learning model comprises training the machine learning model by using the one or more weather conditions of the environment as one or more of the ground truth output labels for the machine learning model,
wherein the machine learning model is configured to predict, based on the target acoustic data, that the one or more weather conditions are present in the environment.

6. The method of claim 1 , wherein training the machine learning model comprises training the machine learning model by using one or more weather conditions of the environment identified from the first data as one or more of the ground truth output labels for the machine learning model,
wherein the machine learning model is configured to predict, based on the target acoustic data, that the one or more weather conditions are present in the environment.

7. The method of claim 1 , wherein the processor is part of a computing device that is positioned remotely from the aircraft.

8. The method of claim 1 , wherein the processor is part of a computing system located onboard the aircraft.

9. A method comprising:
causing an array of microphones arranged on an aircraft to acquire target acoustic data associated with a target object within an environment of the aircraft while the aircraft is in motion; and
executing, by a processor, a machine learning model to predict, based on the target acoustic data, one or more of an azimuth of the target object, a range of the target object, an elevation of the target object, or a type of the target object,
wherein the machine learning model was trained by (i) causing one or more sensors arranged on the aircraft to acquire first data associated with a first object that was within the environment of the aircraft while the aircraft was in motion, wherein the one or more sensors include one or more of a light detection and ranging sensor, a radar sensor, or a camera, (ii) causing the array of microphones arranged on the aircraft to acquire, over approximately the same window of time as the first data was acquired, first acoustic data associated with the first object while the aircraft was in motion, (iii) the processor using the first acoustic data as an input value to the machine learning model, and (iv) the processor using one or more of an azimuth of the first object, a range of the first object, an elevation of the first object, or a type of the first object identified from the first data as ground truth output labels for the machine learning model.

10. The method of claim 9 , further comprising:
based on one or more of the azimuth of the target object, the range of the target object, the elevation of the target object, or the type of the target object, causing the aircraft to move and avoid the target object.

11. The method of claim 9 , wherein causing the array of microphones arranged on the aircraft to acquire the target acoustic data associated with the target object comprises causing one or more microphones arranged on each wing of the aircraft to acquire the target acoustic data associated with the target object.

12. The method of claim 9 , wherein the azimuth of the first object, the range of the first object, and the elevation of the first object identified from the first data are relative to the aircraft, and
wherein the machine learning model was further trained by:
detecting, by the processor, over approximately the same window of time as the first data and the first acoustic data were acquired, automatic dependent surveillance-broadcast information broadcast by the first object, wherein

Assignees

• Boeing Co

Inventors

• Staudinger Tyler C
• Evans Nick S

Classifications

• G01S13/862

  Combination of radar systems with sonar systems · CPC title

• G01W1/00

  Meteorology · CPC title

• G01S19/14

  specially adapted for specific applications · CPC title

• G01S17/933

  of aircraft or spacecraft · CPC title

• G01S17/10

  using transmission of interrupted, pulse-modulated waves (determination of distance by phase measurements G01S17/32) · CPC title