Systems and methods for predicting probabilities of problems at service providers, based on changes implemented at the service providers

Patent metadata

Field	Value
Publication number	US-2019108465-A1
Application number	US-201816154223-A
Country	US
Kind code	A1
Filing date	Oct 8, 2018
Priority date	Oct 9, 2017
Publication date	Apr 11, 2019
Grant date	—

Abstract

Systems and methods are provided for predicting a probability of a problem in a service at a service provider based on implementation of a change to the service. One exemplary method includes a risk engine accessing change records for historical changes in services associated with the service provider where each record includes a text description of the implemented change and a problem/no problem result for the change. For each record, the risk engine normalizes the text description of the implemented change and generates a word-count matrix based on the normalized text description. The risk engine then performs a regression analysis of the generated word-count matrices for the records and the corresponding problem/no problem results, thereby providing a regression model, and generates a predictive algorithm based on a score provided from the regression model and at least one change factor associated with the change records.

First claim

What is claimed is:

1 . A computer-implemented method for use in providing a probability of a problem at a provider based on implementation of a change at the provider, the method comprising:
accessing, by a risk engine computing device, a training data set for changes associated with a provider, the training data set including multiple records, each record including a text description of an implemented change included in the record, one or more change factors associated with the change and a problem/no problem result for the change; for each record in the training data set:
normalizing, by the risk engine computing device, the text description of the implemented change included in the record; and
generating, by the risk engine computing device, a word-count matrix based on the normalized text description;
for the training data set, performing, by the risk engine computing device, a regression analysis of the generated word-count matrices for the multiple records and the problem/no problem results included in the records, thereby providing a regression model based on the text descriptions of the implemented changes included in the multiple records of the training data set; and generating and storing, by the risk engine computing device, a predictive risk model, using a classifier algorithm, based on a score provided from the regression model and at least one of the one or more change factors included in at least one of the multiple records of the training data set, whereby the risk engine computing device is permitted to predict, via the predictive risk model, a probability of a problem in a service provided by a provider in response to implementation of a planned change to the service, based on a text description of the planned change and the at least one of the one or more change factors for the planned change.

2 . The computer-implemented method of claim 1 , wherein normalizing the text description of the implemented change includes cleaning, by the risk engine computing device, numbers, non-English language words, special characters, and/or symbols from the text description.

3 . The computer-implemented method of claim 2 , wherein normalizing the text description further includes removing stop words, included in a stop word list, in a memory of the risk engine computing device, from the text description.

4 . The computer-implemented method of claim 3 , wherein normalizing the text description further includes stemming words included in the text description, when the words in the text description include non-root words.

5 . The computer-implement method of claim 1 , wherein generating the word-count matrix includes generating a term frequency-inverse document frequency (tf-idf) matrix of the normalized text description.

6 . The computer-implement method of claim 5 , wherein performing the regression analysis of the word-count matrices includes performing a logistic regression analysis of the tf-idf matrices generated by the risk engine computing device for the multiple records and the problem/no problem results of the multiple records of the training data set.

7 . The computer-implement method of claim 1 , wherein each of the multiple records in the training set further includes a text description of a reason for the implemented change included in the record;
wherein the method further comprises, for each record in the training data set:
normalizing, by the risk engine computing device, the text description of the reason for the implemented change included in the record; and
generating, by the risk engine computing device, a word-count matrix based on the normalized text description of the reason;
wherein performing the regression analysis includes performing the regression analysis of the generated word-count matrices based on the text descriptions of the implemented changes, the generated word-count matrices for the multiple records based on the text descriptions of the reasons for the implemented changes, and the problem/no problem results included in the records, thereby providing a reason regression model based on the text descriptions of the reasons for the implemented changes included in the multiple records of the training data set; and wherein generating the predictive risk model includes generating the predictive risk model further based on the reason regression model.

8 . The computer-implemented method of claim 1 , further comprising:
receiving a change request for the planned change; and predicting, by the risk engine computing device, via the predictive risk model, the probability of the problem in the service provided by the provider in response to implementation of the planned change, based on the text description of the planned change and the at least one of the one or more change factors for the planned change.

9 . The computer-implement method of claim 8 , wherein the at least one of the one or more change factors includes multiple of: an employee identifier of a user that submitted a change request for the planned change, a risk level of the planned change included in the change request, a type of the planned change, a priority of the planned change, a time duration of the planned change, whether a back out plan exists for the planned change, whether the change request is verified, and whether a test plan exists for the planned change.

10 . A computer-implemented method of providing a probability of a problem at a provider based on implementation of a change at the provider, the method comprising:
receiving a text description of a planned change for a service associated with the provider, a text description of a reason for the planned change, and multiple values for change factors for the planned change; and calculating, by a risk engine computing device, a probability of the planned change causing a problem at the provider, when implemented, based on a predictive risk model derived from a training data set of prior changes to services implemented at the provider.

11 . The computer-implement method of claim 10 , wherein the change factors include multiple of: an employee identifier of a user that submitted a change request for the planned change, a risk level of the planned change included in the change request, a type of the planned change, a priority of the planned change, a time duration of the planned change, whether a back out plan exists for the planned change, whether the change request is verified, and whether a test plan exists for the planned change.

12 . The computer-implemented method of claim 11 , wherein the predictive risk model includes weights for each of the change factors.

13 . The computer-implemented method of claim 11 , further comprising:
normalizing, by the risk engine computing device, the text description of the reason for the planned change; generating, by the risk engine computing device, a term frequency-inverse document frequency (tf-idf) matrix based on the normalized text description of the reason for the planned change; and determining a probability score for the text description of the reason for the planned change, based on a regression analysis of the tf-idf matrix; wherein calculating the probability of the planned change causing a problem at the provider is further based, at least in part, on the determined probability score for the text description of the reason for the planned change.

14 . The computer-implemented method of claim 13 , further comprising:
normalizing, by the risk engine computing device, the text description of the planned change; generating, by the risk engine computing device, a tf-idf matrix based on the normaliz

Assignees

• Mastercard International Inc

Inventors

• Zhou Xianzhe
• Zhang Xiaoying
• Santhapur Meghana

Classifications

• G06Q10/06375

  Prediction of business process outcome or impact based on a proposed change · CPC title

• G06F17/16

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G06Q10/0635 Primary

  Risk analysis of enterprise or organisation activities · CPC title

• G06F17/2785

  Physics · mapped topic