Automatic control loop grading and data labeling

Patent metadata

Field	Value
Publication number	US-11215955-B2
Application number	US-201916696248-A
Country	US
Kind code	B2
Filing date	Nov 26, 2019
Priority date	Nov 26, 2019
Publication date	Jan 4, 2022
Grant date	Jan 4, 2022

Abstract

Concepts and technologies disclosed herein are directed to automated control loop grading and data labeling (“ACLGDL”). An ACLGDL system analyzes results of an execution, by a control loop system, of a control loop. The ACLGDL system can grade the results. The ACLGDL system also can instruct, based at least in part upon the grade of the results of the execution, the control loop system to collect additional data. The ACLGDL system can label the additional data for use by an output system. The ACLGDL system can establish plurality of policies including a grading-analysis policy, a grading-results policy, a labeling-collection policy, a labeling policy, a publishing policy, and a notification policy. The ACLGDL system can publish the data labeled in accordance with the labeling policy based, at least in part, upon the publishing policy. The ACLGDL system can notify the output system based, at least in part, upon the notification policy.

First claim

The invention claimed is:

1. A method comprising:
defining, by an automated control loop grading and data labeling (“ACLGDL”) system executing, via a processor, a plurality of policies;
activating the ACLGDL system for a control loop to be executed by a control loop system, wherein the control loop uses data collected from a domain;
determining, by the ACLGDL system, that the control loop system has executed the control loop and has generated a result;
analyzing, by the ACLGDL system, the result of the control loop based upon a grading-analysis policy that defines how the ACLGDL system is to evaluate a performance of the control loop;
grading, by the ACLGDL system, the result of the control loop based upon a grading-results policy that defines how the ACLGDL system is to determine a grade for the performance of the control loop;
instructing, by the ACLGDL system, the control loop system to collect additional data from the domain based upon a labeling-collection policy that defines the additional data to be collected;
preparing, by the ACLGDL system, the additional data for labeling based upon a labeling assembly policy that defines how the additional data is to be aggregated, organized, and formatted for labeling;
labeling, by the ACLGDL system, the additional data for use by an output system based upon a labeling policy that defines how the additional data is to be labeled, thereby creating labeled data; and
publishing, by the ACLGDL system, to the output system, the grade for the performance of the control loop and the labeled data based upon a publishing policy that defines how the labeled data is to be published.

2. The method of claim 1 , wherein the output system comprises a machine learning system or a system operated by a user.

3. The method of claim 1 , wherein the labeled data comprises a label indicative of a failure; and further comprising proactively notifying, by the ACLGDL system, a mitigation entity of the label indicative of the failure based upon a notification policy that defines the mitigation entity.

4. The method of claim 1 , wherein the domain comprises a cloud computing platform.

5. The method of claim 1 , further comprising determining whether or not to deactivate the ACLGDL.

6. The method of claim 5 , wherein determining to deactivate the ACLGDL is based upon the grade for the performance of the control loop achieving an acceptable grade.

7. The method of claim 5 , wherein determining not to deactivate the ACLGDL is based upon the grade for the performance of the control loop not achieving an acceptable grade.

8. The method of claim 7 , further comprising activating the ACLGDL system for at least one additional control loop to be executed by the control loop system.

9. A computer-readable storage medium comprising computer-executable instructions that, when executed by a processor, cause the processor to perform operations comprising:
defining a plurality of policies;
activating an automated control loop grading and data labeling (“ACLGDL”) system for a control loop to be executed by a control loop system, wherein the control loop uses data collected from a domain;
determining that the control loop system has executed the control loop and has generated a result;
analyzing the result of the control loop based upon a grading-analysis policy that defines how the ACLGDL system is to evaluate a performance of the control loop;
grading the result of the control loop based upon a grading-results policy that defines how the ACLGDL system is to determine a grade for the performance of the control loop;
instructing the control loop system to collect additional data from the domain based upon a labeling-collection policy that defines the additional data to be collected;
preparing the additional data for labeling based upon a labeling assembly policy that defines how the additional data is to be aggregated, organized, and formatted for labeling;
labeling the additional data for use by an output system based upon a labeling policy that defines how the additional data is to be labeled, thereby creating labeled data; and
publishing, to the output system, the grade for the performance of the control loop and the labeled data based upon a publishing policy that defines how the labeled data is to be published.

10. The computer-readable storage medium of claim 9 , wherein the output system comprises a machine learning system or a system operated by a user.

11. The computer-readable storage medium of claim 9 , wherein the labeled data comprises a label indicative of a failure; and further comprising proactively notifying a mitigation entity of the label indicative of the failure based upon a notification policy that defines the mitigation entity.

12. The computer-readable storage medium of claim 9 , wherein the domain comprises a cloud computing platform.

13. The computer-readable storage medium of claim 9 , wherein the operations further comprise determining whether or not to deactivate the ACLGDL.

14. The computer-readable storage medium of claim 13 , wherein determining to deactivate the ACLGDL is based upon the grade for the performance of the control loop achieving an acceptable grade.

15. The computer-readable storage medium of claim 13 , wherein determining not to deactivate the ACLGDL is based upon the grade for the performance of the control loop not achieving an acceptable grade.

16. The computer-readable storage medium of claim 15 , wherein the operations further comprise activating the ACLGDL system for at least one additional control loop to be executed by the control loop system.

17. An automated control loop grading and data labeling (“ACLGDL”) system comprising:
a processor; and
a memory comprising computer-executable instructions that, when executed by the processor, cause the processor to perform operations comprising
defining a plurality of policies,
activating the ACLGDL system for a control loop to be executed by a control loop system, wherein the control loop uses data collected from a domain,
determining that the control loop system has executed the control loop and has generated a result,
analyzing the result of the control loop based upon a grading-analysis policy that defines how the ACLGDL system is to evaluate a performance of the control loop,
grading the result of the control loop based upon a grading-results policy that defines how the ACLGDL system is to determine a grade for the performance of the control loop,
instructing the control loop system to collect additional data from the domain based upon a labeling-collection policy that defines the additional data to be collected,
preparing the additional data for labeling based upon a labeling assembly policy that defines how the additional data is to be aggregated, organized, and formatted for labeling,
labeling the additional data for use by an output system based upon a labeling policy that defines how the additional data is to be labeled, thereby creating labeled data, and
publishing, to the output system, the grade for the performance of the control loop and the labeled data based upon a publishing policy that defines how the labeled data is to be published.

18. The ACLGDL system of claim 17 , wherein the operations further comprise determining whether or not to deactivate the ACLGDL.

19. The ACLGDL system of claim 18 , wherein determining to deactivate the ACLGDL is based upon the grade for the performance of the control loop achieving an acceptable grade.

20. The ACLGDL system of claim 18 , w

Assignees

• At & T Ip I Lp

Inventors

• Aaron Jeffrey
• Fan James

Classifications

• G06F18/2155 Primary

  characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling · CPC title

• G05B13/0265 Primary

  the criterion being a learning criterion · CPC title

• G05B24/02

  electric · CPC title

• G06F17/15

  Correlation function computation {including computation of convolution operations (arithmetic circuits for sum of products per se, e.g. multiply-accumulators G06F7/5443; digital filters, e.g. FIR, IIR, adaptive filters H03H17/00)} · CPC title

• G05B23/0275

  Fault isolation and identification, e.g. classify fault; estimate cause or root of failure · CPC title