Method for real-time enhancement of a predictive algorithm by a novel measurement of concept drift using algorithmically-generated features

Patent metadata

Field	Value
Publication number	US-11144834-B2
Application number	US-201514880130-A
Country	US
Kind code	B2
Filing date	Oct 9, 2015
Priority date	Oct 9, 2015
Publication date	Oct 12, 2021
Grant date	Oct 12, 2021

Abstract

A predictive analytics system and method in the setting of multi-class classification are disclosed, for identifying systematic changes in an evaluation dataset processed by a fraud-detection model by examining the time series histories of an ensemble of entities such as accounts. The ensemble of entities is examined and processed both individually and in aggregate, via a set of features determined previously using a distinct training dataset. The specific set of features in question may be calculated from the entity's time series history, and may or may not be used by the model to perform the classification. Certain properties of the detected changes are measured and used to improve the efficacy of the predictive model.

First claim

What is claimed is:

1. A computer-implemented method executed on one or more processors, the method comprising:
selecting at least one attribute of one or more input data records associated with payment card transaction activity;
partitioning one or more datasets according to a partitioning scheme that is based on the at least one attribute to generate a summary statistic of historical data records, the summary statistic of the historical data records being generated based on historical data records over a given fixed time period by populating a feature space with a directed pair of points for a new datum accrued;
generating a set of features based on the historical data records, the set of features based on the historical data records being hidden nodes of a feed-forward neural network and at least one of the historical data records having a known class membership, the set of features being generated according to different class labels for the known class membership;
calculating a set of microscopic derived features for a new input data record or previous input data records, based on the partitioning and the set of features, the set of microscopic derived features being related to the selected at least one attribute and associated with at least one entity defined by the partitioning scheme;
generating a set of macroscopic derived features based on the aggregated set of microscopic derived features calculated for the new and previous input data records based on the partitioning and the set of features, the set of macroscopic derived features representing concept drift of the one or more datasets;
generating a first score representing a likelihood of an extent and magnitude of the concept drift representing a systematic shift in a property of the one or more input data records over a course of time;
generating a second score representing a likelihood that a new input data record is subject to the extent and magnitude of the concept drift representing by the first score; and
adjusting a class membership for the new input data record using the feed-forward neural network according to at least the magnitude of the concept drift, including a binary classification for fraud detection, the feed-forward neural network being trained using backpropagation and stochastic gradient decent on tagged input data records involving at least a first transaction and a second transaction associated with a cardholder, the second transaction being prior in time to the first transaction, the adjusting improving detection of fraud in payment card transactions by determining the degree with which the payment card transaction activity for the cardholder is unusual with respect to historical cardholder behavior.

2. The method in accordance with claim 1 , wherein the new input data records and the historical data records are mutually exclusive.

3. The method in accordance with claim 1 , wherein the class membership adjustment is trained according to the partitioning scheme.

4. The method in accordance with claim 1 , wherein the class membership adjustment is trained according to the historical data records.

5. The method in accordance with claim 1 , wherein the class membership adjustment includes a binary classification for fraud detection.

6. The method in accordance with claim 1 , wherein partitioning the one or more data records includes generating a summary statistic of the historical data records prior to generating the set of features based on the historical data records.

7. A computer program product comprising a non-transitory machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
selecting at least one attribute of one or more input data records associated with payment card transaction activity;
partitioning one or more datasets according to a partitioning scheme that is based on the at least one attribute to generate a summary statistic of historical data records, the one or more datasets having at least one input data record;
generating a set of features based on historical data records, the set of features based on the historical data records being hidden nodes of a feed-forward neural network and at least one of the historical data records having a known class membership, the set of features being generated according to different class labels for the known class membership;
calculating a set of microscopic derived features for a new input data record or previous input data records, based on the partitioning and the set of features, the set of microscopic derived features being related to the selected at least one attribute;
generating a set of macroscopic derived features based on the aggregated set of microscopic derived features calculated for the new and previous input data records based on the partitioning and the set of features, the set of macroscopic derived features representing a concept drift of the one or more datasets, the concept drift comprising at least a systematic shift in a property of the one or more data records;
generating a first score representing a likelihood of an extent and magnitude of the concept drift representing a systematic shift in a property of the one or more input data records over a course of time;
generating, using the set of microscopic derived features and the set of macroscopic derived features for previous input data records, a second score representing a likelihood that a new input data record is subject to the extent and magnitude of the concept drift representing by the first score;
adjusting a class membership for the new input data record using the feed-forward neural network according to the partitioning scheme based on the second score and the magnitude of the concept drift, including a binary classification for fraud detection; and
determining the degree with which the payment card transaction activity for a cardholder is unusual with respect to historical cardholder behavior.

8. The computer program product in accordance with claim 7 , wherein the microscopic derived features are associated with entity defined by the partitioning scheme and wherein the new input data records and the historical data records are mutually exclusive.

9. The computer program product in accordance with claim 7 , wherein the class membership adjustment is trained according to the partitioning scheme.

10. The computer program product in accordance with claim 7 , wherein the class membership adjustment is trained according to the historical data records.

11. The computer program product in accordance with claim 7 , wherein the class membership adjustment includes a binary classification for fraud detection.

12. The computer program product in accordance with claim 7 , wherein partitioning the one or more data records includes generating a summary statistic of the historical data records prior to generating the set of features based on the historical data records.

13. A system comprising:
at least one programmable processor; and
a non-transitory, machine-readable medium storing instructions that, when executed by the at least one processor, cause the at least one programmable processor to perform operations comprising:
select at least one attribute of one or more input data records associated with payment card transaction activity;
partition one or more datasets according to a partitioning scheme that is based on the at least one attribute to generate a summary statistic of historical data records, the one or more datasets having at least one input data record;
generate a set of features based on historical da

Assignees

• Fair Isaac Corp

Inventors

• Zoldi Scott Michael
• Coggeshall Jim
• Jia Yuting

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N5/02 Primary

  Knowledge representation; Symbolic representation · CPC title

• G06F17/18

  for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)} · CPC title

• G06N20/00

  Machine learning · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title