System and method for prediction using synthetic features and gradient boosted decision tree

Patent metadata

Field	Value
Publication number	US-2017213280-A1
Application number	US-201615007593-A
Country	US
Kind code	A1
Filing date	Jan 27, 2016
Priority date	Jan 27, 2016
Publication date	Jul 27, 2017
Grant date	—

Abstract

A machine learning system and method are disclosed in which a plurality of synthetic features are created from input data, and a gradient boosted decision tree algorithm is then executed by the computer to process both the synthetic features and at least some of the input data to produce an output that is a probability.

First claim

1 . A computer-implemented method comprising:
the computer obtaining a set of data relating to a loan application; the computer determining a plurality of synthetic features by at least: executing a plurality of machine learning algorithms, each of the machine learning algorithms, when executed, receiving as an input at least some of the data and producing as an output a respective synthetic feature representing an initial probability of whether a loan default will occur; the computer executing a gradient boosted decision tree (GBDT) algorithm to process both the synthetic features and at least some of the data to produce an output representing a final probability of whether the loan default will occur.

2 . The computer-implemented method of claim 1 , further comprising:
the computer generating an indication of whether or not to approve the loan based on whether a particular value is above or below a stored threshold; wherein the particular value is based on the final probability.

3 . The computer-implemented method of claim 1 , wherein the plurality of machine learning algorithms are a first set of machine learning algorithms, and wherein determining the plurality of synthetic features further comprises:
the computer executing at least one other machine learning algorithm different from the first set of machine learning algorithms, the at least one other machine learning algorithm, when executed, receiving an input based on at least some of the data and producing a plurality of outputs; wherein each one of the plurality of outputs is a synthetic feature representing a probability of an event occurring, the event being different from the loan default that is associated with the final probability.

4 . The computer-implemented method of claim 3 , wherein the event comprises late payment of the loan.

5 . The computer-implemented method of claim 1 , further comprising:
the computer performing binary encoding of at least some of the data to produce binary encoded data; and the computer inputting the binary encoded data to at least one of the machine learning algorithms.

6 . The computer-implemented method of claim 1 , further comprising:
the computer augmenting the data with at least one additional feature; and the computer inputting the at least one additional feature to at least one of the machine learning algorithms.

7 . The computer-implemented method of claim 1 , wherein the data includes an amount of loan requested and a loan duration, and wherein at least two of the machine learning algorithms are different from each other.

8 . The computer-implemented method of claim 7 , wherein the data further includes a transaction history of the loan applicant, and wherein one of the machine learning algorithms is a neural network that accepts the transaction history as an input.

9 . The computer-implemented method of claim 1 , further comprising:
the computer training the machine learning algorithms, the training including using training data and test data to determine what inputs are to be used for each machine learning algorithm by: the computer trying different possible inputs and selecting a set of one or more inputs that best satisfy a metric.

10 . A system comprising:
a memory to store a set of data relating to a loan application; a predictor to receive the data and to produce an output representing a final probability of whether a loan default will occur; the predictor including a plurality of learners, each learner implementing a respective machine learning algorithm; the predictor configured to:
determine a plurality of synthetic features by sending to each of the learners at least some of the data, and each of the learners outputting a respective synthetic feature representing an initial probability of whether the loan default will occur; and
execute a gradient boosted decision tree (GBDT) algorithm to process both the synthetic features and at least some of the data to produce the output representing the final probability of whether the loan default will occur.

11 . The system of claim 10 , wherein the system is configured to generate an indication of whether or not to approve the loan based on whether a particular value is above or below a stored threshold, wherein the particular value is based on the final probability.

12 . The system of claim 10 , wherein the plurality of learners are a first set of learners, and wherein the predictor is configured to determine the plurality of synthetic features by also:
sending an input to at least one other learner different from the first set of learners, the input based on at least some of the data, and the at least one other learner implementing a machine learning algorithm that, when executed, receives the input and produces a plurality of outputs;
wherein each one of the plurality of outputs is a synthetic feature representing a probability of an event occurring, the event being different from the loan default that is associated with the final probability.

13 . The system of claim 12 , wherein the event comprises late payment of the loan.

14 . The system of claim 10 , wherein the predictor is further configured to:
perform binary encoding of at least some of the data to produce binary encoded data; and send the binary encoded data to at least one of the learners.

15 . The system of claim 10 , wherein the predictor is further configured to:
augment the data with at least one additional feature; and send the at least one additional feature to at least one of the learners.

16 . The system of claim 10 , wherein the data includes an amount of loan requested and a loan duration, and wherein at least two of the learners implement machine learning algorithms that are different from each other.

17 . The system of claim 16 , wherein the data further includes a transaction history of the loan applicant, and wherein one of the learners is a neural network that accepts the transaction history as an input.

18 . The system of claim 10 , wherein the system is configured to train the learners, the training comprising using training data and test data to determine what inputs are to be sent to each learner by: the system trying different possible inputs and selecting a set of one or more inputs that best satisfy a metric.

19 . A system comprising:
at least one processor; and memory having stored thereon processor-executable instructions that, when executed, cause the at least one processor to:
determine a plurality of synthetic features by at least: executing a plurality of machine learning algorithms, each of the machine learning algorithms, when executed, receiving as an input at least some of the data and producing as an output a respective synthetic feature representing an initial probability of whether a loan default will occur, and
execute a gradient boosted decision tree (GBDT) algorithm to process both the synthetic features and at least some of the data to produce an output representing a final probability of whether the loan default will occur.

20 . The system of claim 19 , wherein the processor-executable instructions, when executed, further cause the at least one processor to:
generate an indication of whether or not to approve the loan based on whether a particular value is above or below a stored threshold; wherein the particular value is based on the final probability.

Assignees

• Kaznady Max S
• Huawei Tech Co Ltd

Inventors

• Kaznady Max S

Classifications

• G06Q40/03 Primary

  Credit; Loans; Processing thereof · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N3/02 Primary

  Neural networks · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06Q40/025 Primary

  Physics · mapped topic