Determining the health of an iot application

Patent metadata

Field	Value
Publication number	US-2020052988-A1
Application number	US-201916657184-A
Country	US
Kind code	A1
Filing date	Oct 18, 2019
Priority date	Dec 23, 2016
Publication date	Feb 13, 2020
Grant date	—

Abstract

The example embodiments are directed to a system and method for monitoring the health of analytical applications. In certain embodiments, these analytic applications may be a part of a broader IoT solution that may optionally be hosted on a cloud platform. In one example, the method includes receiving an output of an IoT analytic application deployed on a computing platform, selecting at least one performance metric based on a type of the application, calculating a performance of the application based on the selected performance metric, the received output of the application, and an expected output of the application, and in response to detecting the calculated performance of the application is below a predetermined threshold, outputting an alert to a user device. Accordingly, the health of an IoT analytic application can be monitored and an alert can be provided when the application operates below a predetermined threshold.

First claim

What is claimed is:

1 . A computing system comprising:
a storage configured to store predictions made by an analytic application; a processor configured to
select at least one performance metric for evaluating an accuracy of the analytic application based on a type of the analytic application,
determine an accuracy value of the predictions made by the analytic application based on the selected performance metric, the predictions made by the analytic application, and an expected predictive output of the analytic application, and
determine that a predictive capability of the analytic application has deteriorated based on the determined accuracy value of the predictions; and
a network interface configured to transmit an alert to a user device associated with the analytic application.

2 . The computing system of claim 1 , wherein the processor determines how accurate the predictions made by the analytic application are via execution of the selected at least one performance metric.

3 . The computing system of claim 1 , wherein the accuracy value of the analytic application identifies how accurate the analytic application is for a predetermined period of time.

4 . The computing system of claim 1 , wherein the processor is further configured to display, via a user interface, a list of performance metrics associated with the analytic application that are capable of being selected by a user to measure the predictive accuracy of the analytic application.

5 . The computing system of claim 1 , wherein the at least one performance metric comprises at least one of a receiver operating characteristic (ROC) curve, a confusion matrix, and a lift chart, which compare the predictions made by the analytic application with respect to the expected predictive output of the analytic application.

6 . The computing system of claim 1 , wherein the at least one performance metric comprises at least one of R squared, mean absolute error (MAE), mean squared error (MSE), and mean square root error (RMSE), which compare the predictions made by the analytic application with respect to the expected predictive output of the analytic application.

7 . The computing system of claim 1 , wherein the at least one performance metric comprises a confidence interval plot, which compare the predictions made by the analytic application with respect to the expected predictive output of the analytic application.

8 . The computing system of claim 1 , wherein the processor is further configured to receive new predictions that are made by the analytic application, and determine a new accuracy value of the newly received predictions made by the analytic application based on the selected performance metric, the newly received predictions made by the analytic application, and the expected predictive output of the analytic application.

9 . A method comprising:
receiving predictions made by an analytic application; selecting at least one performance metric for evaluating an accuracy of the analytic application based on a type of the analytic application; determining an accuracy value of the predictions made by the analytic application based on the selected performance metric, the predictions made by the analytic application, and an expected predictive output of the analytic application; determining that a predictive capability of the analytic application has deteriorated based on the determined accuracy value of the predictions; and transmitting an alert to a user device associated with the analytic application.

10 . The method of claim 9 , wherein the determining comprises determining how accurate the predictions made by the analytic application are via execution of the selected at least one performance metric.

11 . The method of claim 9 , wherein the accuracy value of the analytic application identifies how accurate the analytic application is for a predetermined period of time.

12 . The method of claim 9 , further comprising displaying, via a user interface, a list of performance metrics associated with the analytic application that are capable of being selected by a user to measure the predictive accuracy of the analytic application.

13 . The method of claim 9 , wherein the at least one performance metric comprises at least one of a receiver operating characteristic (ROC) curve, a confusion matrix, and a lift chart, which compare the predictions made by the analytic application with respect to the expected predictive output of the analytic application.

14 . The method of claim 9 , wherein the at least one performance metric comprises at least one of R squared, mean absolute error (MAE), mean squared error (MSE), and mean square root error (RMSE), which compare the predictions made by the analytic application with respect to the expected predictive output of the analytic application.

15 . The method of claim 9 , wherein the at least one performance metric comprises a confidence interval plot, which compare the predictions made by the analytic application with respect to the expected predictive output of the analytic application.

16 . The method of claim 9 , further comprising receiving new predictions that are made by the analytic application, and determining a new accuracy value of the newly received predictions made by the analytic application based on the selected performance metric, the newly received predictions made by the analytic application, and the expected predictive output of the analytic application.

17 . A non-transitory computer-readable medium storing instructions which when executed by a processor cause a computer to perform a method comprising:
receiving predictions made by an analytic application; selecting at least one performance metric for evaluating an accuracy of the analytic application based on a type of the analytic application; determining an accuracy value of the predictions made by the analytic application based on the selected performance metric, the predictions made by the analytic application, and an expected predictive output of the analytic application; determining that a predictive capability of the analytic application has deteriorated based on the determined accuracy value; and transmitting an alert to a user device associated with the analytic application.

18 . The non-transitory computer-readable medium of claim 17 , wherein the determining comprises determining how accurate the predictions made by the analytic application are via execution of the selected at least one performance metric.

19 . The non-transitory computer-readable medium of claim 17 , wherein the accuracy value of the analytic application identifies how accurate the analytic application is for a predetermined period of time.

20 . The non-transitory computer-readable medium of claim 17 , wherein the method further comprises displaying, via a user interface, a list of performance metrics associated with the analytic application that are capable of being selected by a user to measure the predictive accuracy of the analytic application.

Assignees

• Gen Electric

Inventors

• Gidugu Roopa
• Aluri Soujanya
• Wang Chris J
• Vakeel Priyanka

Classifications

• H04W4/70

  Services for machine-to-machine communication [M2M] or machine type communication [MTC] · CPC title

• H04L43/16

  Threshold monitoring · CPC title

• H04L67/12

  specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks · CPC title

• H04L43/0817 Primary

  by checking functioning · CPC title

• H04L41/0631

  using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis · CPC title