Audio-based device fault detection method, electronic device and storage medium

Patent metadata

Field	Value
Publication number	US-12353202-B2
Application number	US-202418893399-A
Country	US
Kind code	B2
Filing date	Sep 23, 2024
Priority date	Dec 4, 2023
Publication date	Jul 8, 2025
Grant date	Jul 8, 2025

Abstract

Provided is an audio-based device fault detection method, an electronic device, and a storage medium, relating to the field of data processing and in particular to technical fields of deep learning and voice technology. The method includes: obtaining initial audio data collected by a drone for a target device; preprocessing the initial audio data to obtain audio data to be detected; performing feature extraction on the audio data to be detected to obtain an audio feature of the audio data to be detected; constructing an information graph based on the audio feature; and obtaining a fault detection result for the target device based on the information graph and a graph neural network model.

First claim

What is claimed is:

1. An audio-based device fault detection method, comprising:
obtaining initial audio data collected by a drone for a target device, wherein the target device is a reactor required for a spinning process, and the initial audio data for the reactor is collected by: controlling the drone to execute an operation plan to obtain the initial audio data, the operation plan comprising inspection time, location information of the reactor, and a requirement for the collected initial audio data;
preprocessing the initial audio data to obtain audio data to be detected;
performing feature extraction on the audio data to be detected to obtain an audio feature of the audio data to be detected;
constructing an information graph based on the audio feature;
obtaining a fault detection result for the target device based on the information graph and a graph neural network model;
obtaining, based on collection time of the initial audio data, an operating parameter of the reactor in a preset time range with the collection time as a benchmark, in a case of the fault detection result indicates that the reactor has a fault, wherein the operating parameter comprises at least one of: voltage, current, rotation speed or viscosity;
determining a fault condition of the reactor based on the operating parameter; and
issuing emergency alarm information after determining that the reactor is abnormal based on the operating parameter.

2. The method of claim 1 , wherein the preprocessing the initial audio data to obtain audio data to be detected, comprises:
processing the initial audio data based on noise suppression and a silence detection method to obtain first audio data;
performing frame division on the first audio data to obtain second audio data;
processing the second audio data using a window function to obtain third audio data; and
converting the third audio data into a frequency domain to obtain the audio data to be detected.

3. The method of claim 1 , wherein the obtaining a fault detection result for the target device based on the information graph and a graph neural network model, comprises:
constructing an adjacency matrix based on the information graph; and
inputting the adjacency matrix into the graph neural network model to obtain the fault detection result for the target device.

4. The method of claim 1 , wherein the initial audio data comprises a plurality of audio segments, and the constructing an information graph based on the audio feature comprises:
determining similarities between audio segments based on audio sub-features of the audio segments in the audio feature; and
taking audio segments with similarities meeting a preset threshold as nodes with an associated relationship to construct the information graph.

5. The method of claim 1 , wherein the audio feature comprises at least one of: frequency feature, duration feature, or intensity feature.

6. The method of claim 1 , wherein the constructing an information graph based on the audio feature comprises:
obtaining, based on collection time of the initial audio data, an operating parameter of the reactor in a preset time range with the collection time as a benchmark, wherein the operating parameter comprises at least one of: voltage, current, rotation speed or viscosity; and
constructing the information graph based on the audio feature and the operating parameter.

7. An electronic device, comprising:
at least one processor; and
a memory connected in communication with the at least one processor;
wherein the memory stores an instruction executable by the at least one processor, and the instruction, when executed by the at least one processor, enables the at least one processor to execute:
obtaining initial audio data collected by a drone for a target device, wherein the target device is a reactor required for a spinning process, and the initial audio data for the reactor is collected by: controlling the drone to execute an operation plan to obtain the initial audio data, the operation plan comprising inspection time, location information of the reactor, and a requirement for the collected initial audio data;
preprocessing the initial audio data to obtain audio data to be detected;
performing feature extraction on the audio data to be detected to obtain an audio feature of the audio data to be detected;
constructing an information graph based on the audio feature;
obtaining a fault detection result for the target device based on the information graph and a graph neural network model;
obtaining, based on collection time of the initial audio data, an operating parameter of the reactor in a preset time range with the collection time as a benchmark, in a case of the fault detection result indicates that the reactor has a fault, wherein the operating parameter comprises at least one of: voltage, current, rotation speed or viscosity;
determining a fault condition of the reactor based on the operating parameter; and
issuing emergency alarm information after determining that the reactor is abnormal based on the operating parameter.

8. The electronic device of claim 7 , wherein the preprocessing the initial audio data to obtain audio data to be detected, comprises:
processing the initial audio data based on noise suppression and a silence detection method to obtain first audio data;
performing frame division on the first audio data to obtain second audio data;
processing the second audio data using a window function to obtain third audio data; and
converting the third audio data into a frequency domain to obtain the audio data to be detected.

9. The electronic device of claim 7 , wherein the obtaining a fault detection result for the target device based on the information graph and a graph neural network model, comprises:
constructing an adjacency matrix based on the information graph; and
inputting the adjacency matrix into the graph neural network model to obtain the fault detection result for the target device.

10. The electronic device of claim 7 , wherein the initial audio data comprises a plurality of audio segments, and the constructing an information graph based on the audio feature comprises:
determining similarities between audio segments based on audio sub-features of the audio segments in the audio feature; and
taking audio segments with similarities meeting a preset threshold as nodes with an associated relationship to construct the information graph.

11. The electronic device of claim 7 , wherein the audio feature comprises at least one of: frequency feature, duration feature, or intensity feature.

12. The electronic device of claim 7 , wherein the constructing an information graph based on the audio feature comprises:
obtaining, based on collection time of the initial audio data, an operating parameter of the reactor in a preset time range with the collection time as a benchmark, wherein the operating parameter comprises at least one of: voltage, current, rotation speed or viscosity; and
constructing the information graph based on the audio feature and the operating parameter.

13. A non-transitory computer-readable storage medium storing a computer instruction thereon, wherein the computer instruction is used to cause a computer to execute:
obtaining initial audio data collected by a drone for a target device, wherein the target device is a reactor required for a spinning process, and the initial audio data for the reactor is collected by: controlling the drone to execute an operation plan to obtain the initial audio data, the operation plan comprising inspection time, location information of the reactor, and a requirement for the collected i

Assignees

• Zhejiang Hengyi Petrochemical Co Ltd

Inventors

• Peng Xiantao
• Wang Peng
• Qiu Yibo
• Li Dake
• Zhang Junwei
• Cheng Zhangzhong
• Sheng Jianjun

Classifications

• G05B23/027

  Alarm generation, e.g. communication protocol; Forms of alarm · CPC title

• G06F16/683

  using metadata automatically derived from the content · CPC title

• G10L25/18

  the extracted parameters being spectral information of each sub-band · CPC title

• Y02P90/30

  Computing systems specially adapted for manufacturing · CPC title

• G01H11/06

  by electric means · CPC title