Training data generation via reinforcement learning fault-injection

Patent metadata

Field	Value
Publication number	US-2023087837-A1
Application number	US-202117482068-A
Country	US
Kind code	A1
Filing date	Sep 22, 2021
Priority date	Sep 22, 2021
Publication date	Mar 23, 2023
Grant date	—

Abstract

Systems/techniques for generating training data via reinforcement learning fault-injection are provided. A system can access a computing application. In various aspects, the system can train one or more machine learning models based on responses of the computing application to iterative fault-injections determined via reinforcement learning. More specifically, the system can: inject a first fault into the computing application; record a resultant dataset outputted by the computing application in response to the first fault; train the one or more machine learning models on the resultant dataset and the first fault; compute a reinforcement learning reward based on performance metrics of the one or more machine learning models and based on a quantity of the resultant dataset; update, via execution of a reinforcement learning algorithm, the fault-injection policy based on the reinforcement learning reward; and inject a second fault into the computing application, based on the updated fault-injection policy.

First claim

What is claimed is:

1 . A system, comprising:
a processor that executes computer-executable components stored in a computer-readable memory, the computer-executable components comprising:
a transceiver component that accesses a computing application; and
a training component that trains one or more machine learning models based on responses of the computing application to iterative fault-injections that are determined via reinforcement learning.

2 . The system of claim 1 , wherein the computer-executable components further comprise:
a fault-injection component that injects a first fault into the computing application, based on a fault-injection policy.

3 . The system of claim 2 , wherein the computer-executable components further comprise:
a logging component that records a resultant dataset outputted by the computing application in response to the first fault.

4 . The system of claim 3 , wherein the training component trains the one or more machine learning models on the resultant dataset and the first fault.

5 . The system of claim 4 , wherein the computer-executable components further comprise:
a reward component that evaluates one or more performance metrics of the one or more machine learning models after training, that evaluates a quantity of the resultant dataset, and that computes a reinforcement learning reward based on the one or more performance metrics and the quantity.

6 . The system of claim 5 , wherein the computer-executable components further comprise:
an update component that updates, via execution of a reinforcement learning algorithm, the fault-injection policy based on the reinforcement learning reward.

7 . The system of claim 6 , wherein the fault-injection component injects a second fault into the computing application, based on the updated fault-injection policy.

8 . A computer-implemented method, comprising:
accessing, by a device operatively coupled to a processor, a computing application; and training, by the device, one or more machine learning models based on responses of the computing application to iterative fault-injections that are determined via reinforcement learning.

9 . The computer-implemented method of claim 8 , wherein the training the one or more machine learning models based on responses of the computing application to iterative fault-injections includes:
injecting, by the device, a first fault into the computing application, based on a fault-injection policy.

10 . The computer-implemented method of claim 9 , wherein the training the one or more machine learning models based on responses of the computing application to iterative fault-injections further includes:
recording, by the device, a resultant dataset outputted by the computing application in response to the first fault.

11 . The computer-implemented method of claim 10 , wherein the training the one or more machine learning models based on responses of the computing application to iterative fault-injections further includes:
training, by the device, the one or more machine learning models on the resultant dataset and the first fault.

12 . The computer-implemented method of claim 11 , wherein the training the one or more machine learning models based on responses of the computing application to iterative fault-injections further includes:
evaluating, by the device, one or more performance metrics of the one or more machine learning models after training; evaluating, by the device, a quantity of the resultant dataset; and computing, by the device, a reinforcement learning reward based on the one or more performance metrics and the quantity.

13 . The computer-implemented method of claim 12 , wherein the training the one or more machine learning models based on responses of the computing application to iterative fault-injections further includes:
updating, by the device and via execution of a reinforcement learning algorithm, the fault-injection policy based on the reinforcement learning reward.

14 . The computer-implemented method of claim 13 , wherein the training the one or more machine learning models based on responses of the computing application to iterative fault-injections further includes:
injecting, by the device, a second fault into the computing application, based on the updated fault-injection policy.

15 . A computer program product for facilitating training data generation via reinforcement learning fault-injection, the computer program product comprising a computer-readable memory having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
access, by the processor, a computing application; and train, by the processor, one or more machine learning models based on responses of the computing application to iterative fault-injections that are determined via reinforcement learning.

16 . The computer program product of claim 15 , wherein the processor trains the one or more machine learning models based on responses of the computing application to iterative fault-injections by:
injecting, by the processor, a first fault into the computing application, based on a fault-injection policy.

17 . The computer program product of claim 16 , wherein the processor trains the one or more machine learning models based on responses of the computing application to iterative fault-injections by:
recording, by the processor, a resultant dataset outputted by the computing application in response to the first fault.

18 . The computer program product of claim 17 , wherein the processor trains the one or more machine learning models based on responses of the computing application to iterative fault-injections by:
training, by the processor, the one or more machine learning models on the resultant dataset and the first fault.

19 . The computer program product of claim 18 , wherein the processor trains the one or more machine learning models based on responses of the computing application to iterative fault-injections by:
evaluating, by the processor, one or more performance metrics of the one or more machine learning models after training; evaluating, by the processor, a quantity of the resultant dataset; and computing, by the processor, a reinforcement learning reward based on the one or more performance metrics and the quantity.

20 . The computer program product of claim 19 , wherein the processor trains the one or more machine learning models based on responses of the computing application to iterative fault-injections by:
updating, by the processor and via execution of a reinforcement learning algorithm, the fault-injection policy based on the reinforcement learning reward.

Assignees

• IBM

Inventors

• Hwang Jinho
• Shwartz Larisa
• Rios Aliaga Jesus Maria
• Bagehorn Frank
• Hussey Stephen James

Classifications

• G06N20/00 Primary

  Machine learning · CPC title

• G06F11/3476 Primary

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06K9/6262

  Physics · mapped topic

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title