Auto-encoder enhanced self-diagnostic components for model monitoring

Patent metadata

Field	Value
Publication number	US-2016155136-A1
Application number	US-201414558700-A
Country	US
Kind code	A1
Filing date	Dec 2, 2014
Priority date	Dec 2, 2014
Publication date	Jun 2, 2016
Grant date	—

Abstract

A diagnostic system for model governance is presented. The diagnostic system includes an auto-encoder to monitor model suitability for both supervised and unsupervised models. When applied to unsupervised models, the diagnostic system can provide a reliable indication on model degradation and recommendation on model rebuild. When applied to supervised models, the diagnostic system can determine the most appropriate model for the client based on a reconstruction error of a trained auto-encoder for each associated model. An auto-encoder can determine outliers among subpopulations of consumers, as well as support model go-live inspections.

First claim

What is claimed is:

1 . A system comprising:
a computer-implemented analytics module that receives transaction data of one or more customers, the analytics module storing a model of transactional behaviors and comparing the transaction data with the model of transactional behaviors to determine a likelihood of a specific transaction or behavior of each of the one or more customers, the analytics module further generating a score representing the likelihood of the specific behavior resembling historical data based on a historical learning of the model; a data extractor for extracting an original data sampling from the transaction data; and a computer-implemented auto-encoder coupled with the analytics module by a computer network that includes the data extractor, the auto-encoder receiving the original data sampling and calculating, using the model of the analytics module, one or more latent variables of the model for reconstructing the original data sampling with a reconstructed data set, the auto-encoder further calculating a reconstruction error for the model utilizing one or more latent variables, the reconstruction error representing a deviation of the reconstructed data record from the original data sampling.

2 . The system in accordance with claim 1 , wherein the auto-encoder is further configured to select, from a plurality of models stored by the analytics module, a best model according to a lowest reconstruction error.

3 . The system in accordance with claim 1 , wherein the model stored by the analytics module is in a go-live state, and wherein the historical learning of the model is in a fixed state.

4 . The system in accordance with claim 1 , wherein one or more customers includes a plurality of subpopulations, and wherein the auto-encoder is further configured to identify an outlier of the reconstruction error associated with at least one of the plurality of subpopulations.

5 . The system in accordance with claim 4 , wherein the auto-encoder is further configured to select, from a plurality of models stored by the analytics module, a best model according to a lowest reconstruction error for the outlier associated with the at least one of the plurality of subpopulations.

6 . The system in accordance with claim 1 , wherein the model is an unsupervised model.

7 . The system in accordance with claim 1 , wherein the model is an supervised model, and wherein the historical learning of the model includes human input data.

8 . A method for implementation by one or more data processors forming part of a least one computing system, the method comprising:
receiving, by at least one data processor, transaction data of one or more customers; comparing, by at least one data processor, the transaction data with a model of transactional behaviors to determine a likelihood of a specific transaction or behavior of each of the one or more customers; generating, by at least one data processor, a score representing the likelihood of the specific behavior resembling historical data based on a historical learning of the model; extracting, by at least one data processor, an original data sampling from the transaction data; and receiving, by at least one data processor, the original data sampling; calculating, by at least one data processor using the model of transactional behaviors, one or more latent variables of the model for reconstructing the original data sampling with a reconstructed data set; and calculating, by at least one data processor, a reconstruction error for the model utilizing one or more latent variables, the reconstruction error representing a deviation of the reconstructed data set from the original data sampling.

9 . The method in accordance with claim 8 , further comprising selecting, by a least one data processor, a best model from a plurality of models according to a lowest reconstruction error.

10 . The method in accordance with claim 8 , wherein the model stored by the analytics module is in a go-live state, and wherein the historical learning of the model is in a fixed state.

11 . The method in accordance with claim 8 , wherein one or more customers includes a plurality of subpopulations, and wherein the method further comprises identifying, by at least one data processor, an outlier of the reconstruction error associated with at least one of the plurality of subpopulations.

12 . The method in accordance with claim 11 , further comprising selecting, by at least one data processor, a best model from a plurality of models according to a lowest reconstruction error for the outlier associated with the at least one of the plurality of subpopulations.

13 . The method in accordance with claim 8 , wherein the model is an unsupervised model.

14 . The method in accordance with claim 8 , wherein the model is an supervised model, and wherein the historical learning of the model includes human input data.

15 . A system comprising:
an analytics module implemented by one or more data processors, the analytics module receiving transaction data of one or more customers and comparing the transaction data with a model of transactional behaviors to determine a likelihood of a specific transaction or behavior of each of the one or more customers, the analytics module further generating a score representing the likelihood of the specific behavior resembling historical data based on a historical learning of the model; a data extractor implemented by one or more data processors for extracting an original data sampling from the transaction data; and an auto-encoder implemented by one or more data processors, the auto-encoder receiving the original data sampling and calculating, in an online state or off-line state and using the model, one or more latent variables of the model for reconstructing the original data sampling with a reconstructed data set, the auto-encoder further calculating a reconstruction error for the model utilizing one or more latent variables, the reconstruction error representing a deviation of the reconstructed data set from the original data sampling.

16 . The system in accordance with claim 15 , wherein the auto-encoder is further configured to select, from a plurality of models stored by the analytics module, a best model according to a lowest reconstruction error.

17 . The system in accordance with claim 15 , wherein the model stored by the analytics module is in a go-live state, and wherein the historical learning of the model is in a fixed state.

18 . The system in accordance with claim 15 , wherein one or more customers includes a plurality of subpopulations, and wherein the auto-encoder is further configured to identify an outlier of the reconstruction error associated with at least one of the plurality of subpopulations.

19 . The system in accordance with claim 18 , wherein the auto-encoder is further configured to select, from a plurality of models stored by the analytics module, a best model according to a lowest reconstruction error for the outlier associated with the at least one of the plurality of subpopulations.

20 . The system in accordance with claim 15 , wherein the model is an unsupervised model.

21 . The system in accordance with claim 15 , wherein the model is an supervised model, and wherein the historical learning of the model includes human input data.

Assignees

• Fair Isaac Corp

Inventors

• Zhang Jun
• Zoldi Scott Michael

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N5/045

  Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06Q30/0201 Primary

  Market modelling; Market analysis; Collecting market data · CPC title