Method and system of mitigating predictive multiplicity in a gradient boosting model

Patent metadata

Field	Value
Publication number	US-2026094072-A1
Application number	US-202418902210-A
Country	US
Kind code	A1
Filing date	Sep 30, 2024
Priority date	Sep 30, 2024
Publication date	Apr 2, 2026
Grant date	—

Abstract

A method and system for mitigating predictive multiplicity in a gradient boosting model (GBM). The method includes generating an empirical parameter set based on an approximation search resulting in a subset that includes candidates of at least one weak learner model (WLM) from a predetermined set of WLMs and training iteratively the empirical parameter set to derive a group filtered from the subset based on at least one from among a model selection (MS) technique and an intermediate ensembles (IE) technique. The method also includes selecting sequentially the at least one WLM from the derived group based on the at least one from among the MS technique and the IE technique; and generating the GBM based on a compilation of the sequentially selected at least one WLM, wherein the generated GBM operates below a minimum predefined disagreement threshold related to assessing predictive multiplicity, thereby mitigating the predictive multiplicity.

First claim

What is claimed is:

1 . A method of mitigating predictive multiplicity in a gradient boosting model (GBM), the method being implemented by at least one processor, the method comprising:
generating an empirical parameter set based on an approximation search related to a full parameter set resulting in a subset comprising candidates of at least one weak learner model (WLM) from a predetermined set of WLMs; training iteratively the empirical parameter set to derive a group filtered from the subset based on at least one from among a model selection (MS) technique via reweighted loss and an aggregation technique via an intermediate ensembles (IE) technique, wherein the group comprises the at least one WLM operating with at least one from among a minimal error from the subset and an additive weighted sum; selecting sequentially the at least one WLM from the derived group based on the at least one from among the MS technique and the IE technique; and generating the GBM based on a compilation of the sequentially selected at least one WLM, wherein the generated GBM operates below a minimum predefined disagreement threshold related to assessing predictive multiplicity, thereby mitigating the predictive multiplicity in the GBM.

2 . The method of claim 1 , wherein the MS technique comprises:
computing the reweighted loss for each of the at least one WLM within the subset, wherein the reweighted loss comprises a predefined loss function evaluated at a data sample for the each of the at least one WLM and evaluated at a predefined mean loss function; generating the derived group that collectively operates with a minimum error as indicated by a minimum value of the computed reweighted loss; and choosing the at least one WLM from the derived group, wherein the chosen at least one WLM individually operates with the minimum value of the computed reweighted loss as compared with other WLMs within the derived group.

3 . The method of claim 2 , wherein the MS technique further comprises:
computing residuals for a next training iteration based on the chosen at least one WLM; and returning the derived group at a last gradient boosting iteration of the training iteration.

4 . The method of claim 1 , wherein the IE technique comprises:
constructing at least one weighted ensemble of WLMs based on randomly selecting the at least one WLM from the subset at each of the training iterations; computing an additive weighted sum of the at least one weighted ensemble of WLM based on at least one output from the randomly selected at least one WLM; and generating the derived group based on the computed additive weighted sum.

5 . The method of claim 1 , wherein the full parameter set comprises a Rashomon set that comprises the predetermined set of WLMs within a hypothesis space with population risks associated with that of a predefined empirical risk minimizer.

6 . The method of claim 5 , wherein the generating the empirical parameter set comprises approximating the Rashomon set with the subset from the predetermined set of WLMs within the hypothesis space; and
wherein the empirical parameter set denotes an empirical Rashomon set.

7 . The method of claim 1 , wherein the minimum predefined disagreement threshold comprises a predefined p-disagreement function with a p value of zero.

8 . The method of claim 1 , further comprising: expanding the iterative training to the predetermined set of WLMs.

9 . A computing apparatus for mitigating predictive multiplicity in a gradient boosting model (GBM), comprising:
a processor; a memory; a display; and a communication interface coupled to each of the processor, the memory, and the display, wherein the processor is configured to: generate an empirical parameter set based on an approximation search related to a full parameter set resulting in a subset comprising candidates of at least one weak learner model (WLM) from a predetermined set of WLMs; train iteratively the empirical parameter set to derive a group filtered from the subset based on at least one from among a model selection (MS) technique via reweighted loss and an aggregation technique via an intermediate ensembles (IE) technique, wherein the group comprises the at least one WLM operating with at least one from among a minimal error from the subset and an additive weighted sum; select sequentially the at least one WLM from the derived group based on the at least one from among the MS technique and the IE technique; and generate the GBM based on a compilation of the sequentially selected at least one WLM, wherein the generated GBM operates below a minimum predefined disagreement threshold related to assessing predictive multiplicity, thereby mitigating the predictive multiplicity in the GBM.

10 . The computing apparatus of claim 9 , wherein the MS technique comprises:
computing the reweighted loss for each of the at least one WLM within the subset, wherein the reweighted loss comprises a predefined loss function evaluated at a data sample for the each of the at least one WLM and evaluated at a predefined mean loss function; generating the derived group that collectively operates with a minimum error as indicated by a minimum value of the computed reweighted loss; and choosing the at least one WLM from the derived group, wherein the chosen at least one WLM individually operates with the minimum value of the computed reweighted loss as compared with other WLMs within the derived group.

11 . The computing apparatus of claim 10 , wherein the MS technique further comprises:
computing residuals for a next training iteration based on the chosen at least one WLM; and returning the derived group at a last gradient boosting iteration of the training iteration.

12 . The computing apparatus of claim 9 , wherein the IE technique comprises:
constructing at least one weighted ensemble of WLMs based on randomly selecting the at least one WLM from the subset at each of the training iterations; computing an additive weighted sum of the at least one weighted ensemble of WLM based on at least one output from the randomly selected at least one WLM; and generating the derived group based on the computed additive weighted sum.

13 . The computing apparatus of claim 9 , wherein the full parameter set comprises a Rashomon set that comprises the predetermined set of WLMs within a hypothesis space with population risks associated with that of a predefined empirical risk minimizer;
wherein the generating the empirical parameter set comprises approximating the Rashomon set with the subset from the predetermined set of WLMs within the hypothesis space; wherein the empirical parameter set denotes an empirical Rashomon set; and wherein the minimum predefined disagreement threshold comprises a predefined p-disagreement function with a p value of zero.

14 . The computing apparatus of claim 9 , wherein the processor is further configured to expand the iterative training to the predetermined set of WLMs.

15 . A non-transitory computer readable storage medium storing instructions for mitigating predictive multiplicity in a gradient boosting model (GBM), the non-transitory computer readable storage medium comprising executable code which, when executed by a processor, causes the processor to:
generate an empirical parameter set based on an approximation search related to a full parameter set resulting in a subset comprising candidates of at least one weak learner model (WLM) from a predetermined set of WLMs; train iteratively the empirical parameter set to derive a group filtered from the subset based on at least one from

Assignees

• Jpmorgan Chase Bank Na

Inventors

• Brugere Ivan
• Hsu Hsiang
• Sharma Shubham
• Lecue Freddy
• Chen Richard

Classifications

• G06N20/20 Primary

  Ensemble learning · CPC title