Determining the health of an IOT application

Patent metadata

Field	Value
Publication number	US-10491496-B2
Application number	US-201615389615-A
Country	US
Kind code	B2
Filing date	Dec 23, 2016
Priority date	Dec 23, 2016
Publication date	Nov 26, 2019
Grant date	Nov 26, 2019

Abstract

The example embodiments are directed to a system and method for monitoring the health of analytical applications. In certain embodiments, these analytic applications may be a part of a broader IoT solution that may optionally be hosted on a cloud platform. In one example, the method includes receiving an output of an IoT analytic application deployed on a computing platform, selecting at least one performance metric based on a type of the application, calculating a performance of the application based on the selected performance metric, the received output of the application, and an expected output of the application, and in response to detecting the calculated performance of the application is below a predetermined threshold, outputting an alert to a user device. Accordingly, the health of an IoT analytic application can be monitored and an alert can be provided when the application operates below a predetermined threshold.

First claim

What is claimed is:

1. A computing device comprising:
a storage configured to store an expected predictive output of an Internet of Things (IoT) analytic application deployed on a cloud platform;
a processor configured to receive an actual predictive output of the IoT analytic application running on a host platform, select at least one performance metric, from among a plurality of performance metrics, based on a type of the IoT analytic application, determine a predictive accuracy performance value that identifies how accurate the IoT analytic application is at making predictions in association with an industrial asset based on the selected at least one performance metric, the actual predictive output of the IoT analytic application, and the expected predictive output of the IoT analytic application, and determine that a predictive performance of the IoT analytic application running on the host platform has deteriorated based on the determined predictive accuracy performance value,
wherein the processor is configured to dynamically provide the at least one performance metric based on the type of the IoT application including:
at least one of a receiver operating characteristic (ROC) curve, a confusion matrix, and a lift chart, in response to the type of the IoT analytic application being a classification based IoT analytic,
at least one of R squared, mean absolute error (MAE), mean squared error (MSE), and mean square root error (RMSE), in response to the type of the IoT analytic application being a regression based IoT analytic, or
at least one of MAE, MSE, a plot of MAE against a plot of forecast time horizon, and a confidence interval plot, in response to the type of the IoT analytic application being a time-series based IoT analytic, and

the processor is further configured to transmit an alert to a user device indicating that the predictive performance of the IoT analytic application running on the host platform has deteriorated.

2. The computing device of claim 1 , wherein the predictive accuracy performance value identifies how successful the IoT analytic application is at making predications in association with an industrial asset based on a comparison of the received actual predictive output of the IoT analytic application and the expected predictive output of the IoT analytic application using the selected at least one performance metric.

3. The computing device of claim 1 , wherein the processor is further configured to display, via a user interface, a list of performance metrics associated with the IoT analytic application that are capable of being selected by a user.

4. The computing device of claim 1 , wherein the processor is configured to display the list of performance metrics including at least one of the ROC curve, the confusion matrix, and the lift chart, in response to the type of the IoT analytic application being the classification based IoT analytic.

5. The computing device of claim 1 , wherein the processor is configured to display the list of performance metrics including at least one of the R squared, the MAE, the MSE, and the RMSE, in response to the type of the IoT analytic application being the regression based IoT analytic.

6. The computing device of claim 1 , wherein the processor is configured to display the list of performance metrics including at least one of the MAE, the MSE, the plot of MAE against the plot of forecast time horizon, and the confidence interval plot, in response to the type of the IoT analytic application being the time-series based IoT analytic.

7. The computing device of claim 1 , wherein the processor is further configured to receive a new predictive output of the IoT analytic application deployed on the cloud platform, and determine another predictive accuracy performance value of the IoT analytic application based on the previously selected at least one performance metric, the newly received predictive output of the IoT analytic application, and the expected predictive output of the IoT analytic application.

8. A method comprising:
receiving an actual predictive output of an IoT analytic application deployed on a host platform;
selecting at least one performance metric, from among a plurality of performance metrics, based on a type of the IoT analytic application;
determine a predictive accuracy performance value that identifies how accurate the IoT analytic application is at making predictions in association with an industrial asset based on the selected at least one performance metric, the received actual predictive output of the IoT analytic application on the host platform, and an expected predictive output of the IoT analytic application;
determining that a predictive performance of the IoT analytic application on the host platform has deteriorated based on the determined predictive accuracy performance value;
dynamically providing the at least one performance metric based on the type of the IoT application including:
at least one of a receiver operating characteristic (ROC) curve, a confusion matrix, and a lift chart, in response to the type of the IoT analytic application being a classification based IoT analytic,
at least one of R squared, mean absolute error (MAE), mean squared error (MSE), and mean square root error (RMSE), in response to the type of the IoT analytic application being a regression based IoT analytic, or
at least one of MAE, MSE, a plot of MAE against a plot of forecast time horizon, and a confidence interval plot, in response to the type of the IoT analytic application being a time-series based IoT analytic, and

in response, transmitting an alert to a user device indicating that the predictive performance of the IoT analytic application deployed on the host platform has deteriorated.

9. The method of claim 8 , wherein the predictive accuracy performance value identifies how successful the IoT analytic application is at making predictions in association with an industrial asset based on a comparison of the received predictive output of the IoT analytic application and the expected predictive output of the IoT analytic application using the selected at least one performance metric.

10. The method of claim 8 , further comprising displaying, via a user interface, a list of performance metrics associated with the IoT analytic application that are capable of being selected by a user.

11. The method of 10 , wherein the displayed list of performance metrics comprises at least one of the ROC curve, the confusion matrix, and the lift chart, in response to the type of the IoT analytic application being the classification based IoT analytic.

12. The method of claim 10 , wherein the displayed list comprises of performance metrics comprises at least one of the R squared, the MAE, the MSE, and the RMSE, in response to the type of the IoT analytic application being the regression based IoT analytic.

13. The method of claim 10 , wherein the displayed list of performance metrics comprises at least one of MAE, MSE, a plot of MAE against a plot of forecast time horizon, and a confidence interval plot, in response to the type of the IoT analytic application being a time-series based IoT analytic.

14. The method of claim 8 , further comprising receiving a new predictive output of the IoT analytic application deployed on the cloud platform, and
determining another predictive performance value of the IoT analytic application based on the previously selected at least one performance metric, the newly received predictive output of the IoT analytic application, and the expected predictive output of the IoT analytic application.

15. A non-transitory compu

Assignees

• Gen Electric

Inventors

• Gidugu Roopa
• Aluri Soujanya
• Wang Chris J
• Vakeel Priyanka

Classifications

• H04W4/70

  Services for machine-to-machine communication [M2M] or machine type communication [MTC] · CPC title

• H04L41/0631

  using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis · CPC title

• H04L67/12

  specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks · CPC title

• H04L43/16

  Threshold monitoring · CPC title

• H04L43/0817 Primary

  by checking functioning · CPC title