Hierarchical feature selection and predictive modeling for estimating performance metrics

Patent metadata

Field	Value
Publication number	US-11080764-B2
Application number	US-201715458484-A
Country	US
Kind code	B2
Filing date	Mar 14, 2017
Priority date	Mar 14, 2017
Publication date	Aug 3, 2021
Grant date	Aug 3, 2021

Abstract

A bid management system generates estimated performance metrics at the bid unit level to facilitate bid optimization. The bid management system includes a hierarchical feature selection and prediction approach. Feature selection is performed by aggregating historical performance metrics to a higher hierarchical level and testing features for statistical significance. Features for which a significance level satisfies a significance threshold are selected for prediction analysis. The prediction analysis uses a statistical model based on selected features to generate estimated performance metrics at the bid unit level. In some implementations, the prediction analysis uses a hierarchical Bayesian smoothing method in which estimated performance metrics are calculated at the bid unit level using a posterior probability distribution derived from a prior probability distribution determined based on aggregated performance metrics and a likelihood function that takes into account historical performance metrics from the bid unit level based on the selected features.

First claim

What is claimed is:

1. A computer system comprising:
a processor; and
a computer storage medium storing computer-useable instructions that, when used by the processor, cause the processor to:
perform feature selection to generate a machine learning model, the feature selection being done by:
aggregating, by the processor executing a feature selection component, performance metrics of each feature, of a plurality of features, for a plurality of bid units from a bid unit hierarchical level to a first hierarchical level to generate aggregated performance data, the first hierarchical level comprising a hierarchical level above the bid unit hierarchical level within a hierarchical structure;
in response to the aggregating of the performance metrics, employing, by the processor executing the feature selection component, a statistical testing function to determine a significance level for each feature, of the plurality of features, at the first hierarchical level using the aggregated performance data; and
comparing, by the processor executing the feature selection component, the significance level at the first hierarchical level for each feature to a significance threshold;
based on determining that the significance level for a first feature, of the plurality of features, at the first hierarchical level satisfies the significance threshold, generating, by the processor executing the feature selection component, the machine learning model using the first feature as input;
based on determining that the significance level for a second feature, of the plurality of features, at the first hierarchical level fails to satisfy the significance threshold, excluding the second feature from being used as input into the machine learning model;
performing, by the processor executing a prediction component, a prediction analysis using the first feature and the machine learning model based at least in part on the aggregating of the performance metrics, the generating of the machine learning model, and the excluding of the second feature; and
in response to the performing of the prediction analysis, causing, by the processor executing a SEM management UI component, a user interface to be generated, the user interface indicates the feature and the prediction analysis.

2. The system of claim 1 , wherein the hierarchical structure is based on a marketer's search engine marketing portfolio.

3. The system of claim 2 , wherein the first hierarchical level is any of: an ad group level, a campaign level, and a portfolio level.

4. The system of claim 1 , wherein the statistical testing function comprises a parametric or nonparametric statistical test.

5. The system of claim 4 , wherein the statistical testing function comprises analysis of variance (ANOVA) or analysis of similarities.

6. The system of claim 1 , wherein the instructions further cause the processors to calculate adjustment factors using the estimated performance metrics.

7. The system of claim 6 , wherein the instructions further cause the processors to apply the adjustment factors to bid amounts to adjust the bid amounts.

8. A computer storage medium storing computer-useable instructions that, when executed by a computing device, cause the computing device to perform operations, the operations comprising:
performing feature selection to select a feature of a plurality of features for a plurality of bid units by aggregating, by the computing executing a feature selection component, performance metrics for each feature for the plurality of bid units from a bid unit hierarchical level to a first hierarchical level and, the first hierarchical level comprising a hierarchical level above the bid unit hierarchical level within a hierarchical structure;
employing, by the computing device executing the feature selection component, a statistical testing function to determine a significance level for each feature, of the plurality of features, at the first hierarchical level using the aggregated performance statistics;
based on determining that the significance level for a first feature, of the plurality of feature at the first hierarchical level exceeds a threshold and that a second feature, of the plurality of features, fails to exceed the threshold, generating, by the computing device executing the feature selection component, a model with the first feature as input, wherein the second feature is excluded from being used by the model as input;
in response to the generating of the model with the first feature as input, performing, by the computing device executing a prediction component, prediction analysis by using hierarchical Bayesian smoothing to employ the aggregated performance metrics as a prior probability distribution to generate estimated performance metrics for each bid unit.

9. The computer storage medium of claim 8 , wherein the aggregated performance metrics are tested for statistical significance for the feature from the plurality of features by:
comparing the significance level for the feature to a significance threshold; and
selecting the feature for prediction analysis based on the significance level for the feature satisfying the significance threshold.

10. The computer storage medium of claim 8 , wherein the hierarchical structure is based on a marketer's search engine marketing portfolio.

11. The computer storage medium of claim 10 , wherein the first hierarchical level is any of: an ad group level, a campaign level, and a portfolio level.

12. The computer storage medium of claim 8 , wherein the statistical testing function comprises a parametric or nonparametric statistical test.

13. The computer storage medium of claim 12 , wherein the statistical testing function comprises analysis of variance (ANOVA) or analysis of similarities.

14. The computer storage medium of claim 8 , wherein the hierarchical Bayesian smoothing comprises:
determining the prior probability distribution using the aggregated performance metrics;
deriving a posterior probability distribution from the prior probability distribution and a likelihood function using the selected feature and performance metrics for each bid unit; and
calculating the estimated performance metrics for each bid unit using the posterior probability distribution.

15. The computer storage medium of claim 8 , wherein the operations further comprise calculating adjustment factors using the estimated performance metrics.

16. The computer storage medium of claim 15 , wherein the operations further comprise applying the adjustment factors to bid amounts to adjust the bid amounts.

17. A computer system comprising:
feature selection means for performing feature selection to select a first feature as input into a model and to exclude a second feature from the model, the selecting of the first feature and the excluding of the second feature being based on using aggregated performance data generated by aggregating performance metrics for the first feature and second feature for a plurality of bid units from a bid unit hierarchical to a first hierarchical level above the bid unit hierarchical level within a hierarchical structure; and
prediction means for generating estimated performance metrics for each bid unit using the model generated from the performance metrics and the first feature selected by the feature selection based at least in part on the selecting of the first feature and the excluding of the second feature.

18. The computer system of claim 17 , wherein the hierarchical structure is based on a marketer's search engine marketing portfolio.

Assignees

• Adobe Inc

Inventors

• Dong Chen
• Yan Zhenyu
• Panigrahi Pinak
• Wu Xiang
• Pani Abhishek

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06Q30/08

  Auctions · CPC title

• G06F16/245

  Query processing · CPC title

• G06Q30/02

  Marketing; Price estimation or determination; Fundraising · CPC title

• G06Q30/0277 Primary

  Online advertisement · CPC title