Systems and techniques for determining the predictive value of a feature

Patent metadata

Field	Value
Publication number	US-12566993-B2
Application number	US-201916447924-A
Country	US
Kind code	B2
Filing date	Jun 20, 2019
Priority date	May 23, 2014
Publication date	Mar 3, 2026
Grant date	Mar 3, 2026

Abstract

A method for determining the predictive value of a feature may include: (a) performing predictive modeling procedures associated with respective predictive models, wherein performing each modeling procedure includes fitting the associated model to an initial dataset representing an initial prediction problem; (b) determining a first accuracy score of each of the fitted models, representing an accuracy with which the fitted model predicts an outcome of the initial prediction problem; (c) shuffling values of a feature across observations included in the initial dataset, thereby generating a modified dataset representing a modified prediction problem; (d) determining a second accuracy score of each of the fitted models, representing an accuracy with which the fitted model predicts an outcome of the modified prediction problem; and (e) determining a model-specific predictive value of the feature for each of the fitted models based on the first and second accuracy scores of the fitted model.

First claim

What is claimed is:

1 . A computer-implemented predictive modeling method comprising:
(a) performing a predictive modeling procedure, wherein the predictive modeling procedure is associated with a predictive model, wherein performing the modeling procedure comprises fitting the associated predictive model to an initial dataset, and wherein the initial dataset includes a plurality of observations and each observation includes respective values for a plurality of features; (b) determining a model-specific predictive value of a feature for the fitted model; (c) based at least in part on the model-specific predictive value of the feature, generating a modified dataset, wherein generating the modified dataset includes pruning the feature from the initial dataset and/or creating a first derived feature based on the feature and at least one other feature in the initial dataset and adding the first derived feature to the modified dataset; and (d) performing the predictive modeling procedure again, wherein performing the modeling procedure again comprises fitting the associated predictive model to the modified dataset, wherein the feature is a first feature, wherein the initial dataset includes the first feature and a plurality of second features, and wherein the method further comprises detecting an interaction between at least two of the second features, and wherein generating the modified dataset further comprises creating a second derived feature based on the at least two second features and adding the second derived feature to the modified dataset.

2 . The method of claim 1 , wherein the method further comprises determining model-specific predictive values of the second features of the initial dataset.

3 . The method of claim 2 , further comprising displaying, via a graphical user interface, graphical content identifying the first and second features of the initial dataset and the model-specific predictive values of the first and second features.

4 . The method of claim 2 , wherein generating the modified dataset further includes performing feature engineering on the initial dataset based on the model-specific predictive values of the first feature and the second features.

5 . The method of claim 4 , wherein performing the feature engineering comprises:
generating a third derived feature based on two or more features of the initial dataset having high model-specific predictive values; and adding the third derived feature to the modified dataset.

6 . The method of claim 5 , further comprising determining that the model-specific predictive values of the two or more features are high based on the model-specific predictive values of the two or more features being higher than a threshold value and/or based on the model-specific predictive values of the two or more features being in a particular percentile of the particular model-specific predictive values for the first and second features of the initial dataset.

7 . The method of claim 2 , further comprising:
(e) displaying, via a graphical user interface, graphical content ranking the first feature and the plurality of second features based on the respective model-specific predictive values specific to the first predictive model.

8 . The method of claim 1 , further comprising detecting one or more statistically significant interactions between two or more features included in the first and second features, and providing the one or more statistically significant interactions as input to a modeling procedure selected from the group consisting of a random forest modeling procedure, a generalized additive modeling procedure, and a support vector machine modeling procedure.

9 . The method of claim 1 , wherein generating the modified dataset further comprises determining that two or more of the second features are highly correlated with each other and pruning at least one of the two or more highly correlated second features.

10 . The method of claim 1 , wherein the model-specific predictive value of the first feature for the fitted model is determined using an analysis technique applicable to any predictive model.

11 . The method of claim 1 , wherein determining the model-specific predictive value of the first feature for the fitted model comprises analyzing a significance of the first feature in predicting a target based on conditional expectations.

12 . The method of claim 1 , wherein determining the model-specific predictive value of the first feature for the fitted model comprises:
determining a first accuracy score of the fitted predictive model representing an accuracy with which the fitted model predicts one or more outcomes based on the initial dataset; shuffling values of the first feature across respective observations included in the initial dataset, thereby generating a second dataset; and determining a second accuracy score of the fitted predictive model representing an accuracy with which the fitted model predicts one or more outcomes based on the second dataset.

13 . The method of claim 1 , wherein the predictive modeling procedure is a first predictive modeling procedure, the predictive model is a first predictive model, and the model-specific predictive value is a first model-specific predictive value, the method further comprising:
performing at least one second predictive modeling procedure, wherein the second predictive modeling procedure is associated with a second predictive model, wherein performing the second modeling procedure comprises fitting the second predictive model to the initial dataset; and determining a second model-specific predictive value of the first feature for the fitted second model.

14 . The method of claim 13 , further comprising determining a model-independent predictive value of the first feature based on the first and second model-specific predictive values of the first feature.

15 . The method of claim 14 , wherein determining the model-independent predictive value of the first feature comprises calculating a statistical measure of a center and/or a spread of the first and second model-specific predictive values of the first feature.

16 . The method of claim 15 , wherein the statistical measure of the center is selected from the group consisting of a mean, a median, and a mode of the first and second model-specific predictive values, and wherein the statistical measure of the spread is selected from the group consisting of a range, a variance, and a standard deviation of the first and second model-specific predictive values.

17 . The method of claim 14 , wherein determining the model-independent predictive value of the first feature comprises calculating a combination of the first and second model-specific predictive values of the first feature.

18 . The method of claim 14 , wherein generating the modified dataset based at least in part on the model-specific predictive value of the first feature comprises generating the modified dataset based at least in part on the model-independent predictive value of the first feature.

19 . The method of claim 13 , further comprising, prior to performing the at least one second modeling procedure:
selecting the second modeling procedure based on a suitability of the selected modeling procedure for the initial dataset, wherein a suitability of a particular predictive modeling procedure for the initial dataset is determined based, at least in part, on characteristics of one or more particular features of the initial dataset having high model-specific predictive values for t

Assignees

• Datarobot Inc

Inventors

• Achin Jeremy
• Degodoy Thomas
• Conort Xavier
• Yurgenson Sergey
• Steadman Mark L
• Koundry Glen

Classifications

• G06Q30/0201

  Market modelling; Market analysis; Collecting market data · CPC title

• G06Q10/067

  Enterprise or organisation modelling · CPC title

• G06Q10/06

  Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling · CPC title

• G06Q10/04

  Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" (market predictions or forecasting for commercial activities G06Q30/0202) · CPC title

• G06N20/00 Primary

  Machine learning · CPC title