Deep learning of fault detection in onboard automobile systems

Patent metadata

Field	Value
Publication number	US-11989983-B2
Application number	US-202117241481-A
Country	US
Kind code	B2
Filing date	Apr 27, 2021
Priority date	May 7, 2020
Publication date	May 21, 2024
Grant date	May 21, 2024

Abstract

Methods and systems for vehicle fault detection include collecting operational data from sensors in a vehicle. The sensors are associated with vehicle sub-systems. The operational data is processed with a neural network to generate a fault score, which represents a similarity to fault state training scenarios, and an anomaly score, which represents a dissimilarity to normal state training scenarios. The fault score is determined to be above a fault score threshold and the anomaly score is determined to be above an anomaly score threshold to detect a fault. A corrective action is performed responsive the fault, based on a sub-system associated with the fault.

First claim

What is claimed is:

1. A method for vehicle fault detection, comprising:
collecting operational data from a plurality of sensors in a vehicle, the plurality of sensors being associated with a plurality of vehicle sub-systems;
concatenating data features from the operational data with a sensor correlation graph that includes indications of correlations between groups of related sub-systems;
processing the concatenated data with a neural network to generate a fault score, which represents a similarity to fault state training scenarios, and an anomaly score, which represents a dissimilarity to normal state training scenarios;
determining that the fault score is above a fault score threshold and that the anomaly score is above an anomaly score threshold to detect a fault; and
performing a corrective action responsive the fault, based on a sub-system associated with the fault, selected from the group consisting of changing an operational status of one or more of the plurality of vehicle sub-systems, changing the setting of a device in the vehicle, stopping an engine of the vehicle, applying brakes of the vehicle, and changing operational properties of the engine, transmission or brakes of the vehicle to compensate for adverse conditions.

2. The method of claim 1 , wherein collecting the operational data includes extracting data features from raw sensor data using a long-short term memory neural network.

3. The method of claim 1 , wherein at least one node of the sensor correlation graph belongs to more than one group of related sub-systems.

4. The method of claim 1 , wherein the neural network includes a fully connected layer to generate the fault score and an autoencoder to generate the anomaly score.

5. The method of claim 1 , wherein the sensors monitor respective electronic control units of the vehicle to generate respective time series.

6. The method of claim 1 , further comprising receiving parameters of the neural network from a remote neural network training system.

7. A vehicle, comprising:
a plurality of sensors, each associated with a vehicle sub-system;
a hardware processor; and
a memory that stores computer program code, which, when executed by the hardware processor, causes the hardware processor to:
collect operational data from the plurality of sensors;
concatenate data features from the operational data with a sensor correlation graph that includes indications of correlations between groups of related sub-systems;
process the concatenated data with a neural network to generate a fault score, which represents a similarity to fault state training scenarios, and an anomaly score, which represents a dissimilarity to normal state training scenarios;
determine that the fault score is above a fault score threshold and that the anomaly score is above an anomaly score threshold to detect a fault; and
trigger a corrective action responsive the fault, based on a sub-system associated with the fault, selected from the group consisting of changing an operational status of one or more of the plurality of vehicle sub-systems, changing the setting of a device in the vehicle, stopping an engine of the vehicle, applying brakes of the vehicle, and changing operational properties of the engine, transmission or brakes of the vehicle to compensate for adverse conditions.

8. The vehicle of claim 7 , wherein collecting the operational data includes extracting data features from raw sensor data using a long-short term memory neural network.

9. The vehicle of claim 1 , wherein at least one node of the sensor correlation graph belongs to more than one group of related sub-systems.

10. The vehicle of claim 7 , wherein the neural network includes a fully connected layer to generate the fault score and an autoencoder to generate the anomaly score.

11. The vehicle of claim 7 , wherein the sensors monitor respective electronic control units of the vehicle to generate respective time series.

12. The vehicle of claim 7 , further comprising receiving parameters of the neural network from a remote neural network training system.

13. A vehicle, comprising:
a plurality of sensors, each associated with a vehicle sub-system;
a hardware processor; and
a memory that stores computer program code, which, when executed by the hardware processor, causes the hardware processor to:
collect operational data from the plurality of sensors;
concatenate data features from the operational data with a sensor correlation graph that includes indications of correlations between groups of related sub-systems;
process the operational data with a neural network, which includes a fully connected layer to generate a fault score and an autoencoder to generate an anomaly score, wherein the fault score represents a similarity to fault state training scenarios and the anomaly score represents a dissimilarity to normal state training scenarios;
determine that the fault score is above a fault score threshold and that the anomaly score is above an anomaly score threshold to detect a fault; and
trigger a corrective action responsive the fault, based on a sub-system associated with the fault, selected from the group consisting of changing an operational status of one or more of the plurality of vehicle sub-systems, changing the setting of a device in the vehicle, stopping an engine of the vehicle, applying brakes of the vehicle, and changing operational properties of the engine, transmission or brakes of the vehicle to compensate for adverse conditions.

Assignees

• Nec Lab America Inc
• Nec Corp

Inventors

• Tang Luan
• Chen Haifeng
• Cheng Wei
• Rhee Junghwan
• Kamimura Jumpei

Classifications

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G07C5/085 Primary

  using electronic data carriers · CPC title

• B60W50/0205

  Diagnosing or detecting failures; Failure detection models · CPC title