Machine-learning techniques to predict offsite user interactions based on onsite machine- learned models

Patent metadata

Field	Value
Publication number	US-11004108-B2
Application number	US-201916457511-A
Country	US
Kind code	B2
Filing date	Jun 28, 2019
Priority date	Jun 28, 2019
Publication date	May 11, 2021
Grant date	May 11, 2021

Abstract

Techniques for predicting an offsite entity interaction rate are provided. One approach involves using a first machine-learned model that includes a first plurality of features that correspond to entity and campaign attributes. The approach also involves training a second machine-learned model that includes a second plurality of features that includes a particular feature corresponding to predicted entity interaction rates. Thus, output of the first machine-learned model is input to the second machine-learned model. The second machine-learned model includes multiple weights that include a particular weight for the particular feature. A content request is received and a set of campaigns is identified based on an entity identifier associated with the content request. Scores are generated based on the first and second machine-learned models. Based on the scores, a campaign is selected and campaign data associated with the campaign is transmitted over a computer network.

First claim

What is claimed is:

1. A method comprising:
generating, using a first machine-learned model, a first plurality of predicted entity interaction rates, each entity interaction rate corresponding to an entity-campaign pair, wherein the first machine-learned model comprises a first plurality of features that comprises attributes of an entity and attributes of a campaign;
training, using one or more machine learning techniques and the first plurality of predicted entity interaction rates, a second machine-learned model that comprises (1) a second plurality of features that includes a particular feature corresponding to the first plurality of predicted entity interaction rates generated by the first machine-learned model and (2) a plurality of weights that includes a particular weight corresponding to the particular feature;
generating, using the first machine-learned model, a second plurality of predicted entity interaction rates;
after generating the second plurality of predicted entity interaction rates, receiving, over a computer network, from a third-party online exchange, a content request;
in response to receiving the content request:
identifying an entity identifier associated with the content request;
based on the entity identifier, identifying a set of campaigns that is associated with the entity identifier;
for each campaign in the set of campaigns:
identifying a plurality of feature values that correspond to the second plurality of features and that include a predicted entity interaction rate, from the second plurality of predicated entity interaction rates, that corresponds to a pair comprising said each campaign and an entity identified by the entity identifier;
generating, using the second machine-learned model, a score based on the plurality of feature values;
adding the score to a set of scores;

selecting a particular campaign from among the set of campaigns based on the set of scores;
in response to selecting the particular campaign, causing campaign data associated with the particular campaign to be transmitted to the third-party online exchange;

wherein the method is performed by one or more computing devices.

2. The method of claim 1 , wherein the first plurality of features comprises an observed onsite entity interaction rate.

3. The method of claim 1 , wherein the second plurality of features comprises at least one of observed entity interaction rates for one or more campaigns, observed entity interaction rates for one or more third-party exchanges, observed entity interaction rates for one or more third-party publisher systems, observed entity interaction rates for one or more Operating Systems (OS), observed entity interaction rates for one or more campaign formats, or observed entity interaction rates for a particular campaign rendered on a particular third-party publisher systems.

4. The method of claim 1 , wherein generating the score is based at least in part on:
identifying interaction activities associated with the identified entity identifier;
determining that the interaction activities occurred within a threshold number of days;
upon determining that the interaction activities occurred within the threshold number of days, generating the score for the identified entity identifier.

5. The method of claim 1 , further comprising:
identifying a first set of entity identifiers that is associated with a first campaign;
determining that a number of the first set of entity identifiers is below a threshold number;
upon determining that the number of the first set of entity identifiers is below the threshold number, assigning a pre-defined score to the first set of entity identifiers that is associated with the first campaign.

6. The method of claim 1 , further comprising:
identifying a second set of entity identifiers that is associated with a second campaign;
determining that a second set of scores for the second set of entity identifiers falls into a threshold range of score;
upon determining that the second set of scores for the second set of entity identifiers falls into the threshold range of score, disassociating the second campaign from the second set of entity identifiers;
adding a Gaussian random noise value to the second set of scores for the second set of entity identifiers.

7. The method of claim 1 , wherein the score is generated using one or more discretization techniques.

8. The method of claim 1 , wherein the score is generated using one or more histogram techniques.

9. One or more non-transitory storage media storing instructions which, when executed by one or more processors, perform a method comprising:
generating, using a first machine-learned model, a first plurality of predicted entity interaction rates, each entity interaction rate corresponding to an entity-campaign pair, wherein the first machine-learned model comprises a first plurality of features that comprises attributes of an entity and attributes of a campaign;
training, using one or more machine learning techniques and the first plurality of predicted entity interaction rates, a second machine-learned model that comprises (1) a second plurality of features that includes a particular feature corresponding to the first plurality of predicted entity interaction rates generated by the first machine-learned model and (2) a plurality of weights that includes a particular weight corresponding to the particular feature;
generating, using the first machine-learned model, a second plurality of predicted entity interaction rates;
after generating the second plurality of predicted entity interaction rates, receiving, over a computer network, from a third-party online exchange, a content request;
in response to receiving the content request:
identifying an entity identifier associated with the content request;
based on the entity identifier, identifying a set of campaigns that is associated with the entity identifier;
for each campaign in the set of campaigns:
identifying a plurality of feature values that correspond to the second plurality of features and that include a predicted entity interaction rate, from the second plurality of predicated entity interaction rates, that corresponds to a pair comprising said each campaign and an entity identified by the entity identifier;
generating, using the second machine-learned model, a score based on the plurality of feature values;
adding the score to a set of scores;

selecting a particular campaign from among the set of campaigns based on the set of scores;
in response to selecting the particular campaign, causing campaign data associated with the particular campaign to be transmitted to the third-party online exchange.

10. The one or more non-transitory storage media of claim 9 , wherein the first plurality of features comprises an observed onsite entity interaction rate.

11. The one or more non-transitory storage media of claim 9 , wherein the second plurality of features comprises at least one of observed entity interaction rates for one or more campaigns, observed entity interaction rates for one or more third-party exchanges, observed entity interaction rates for one or more third-party publisher systems, observed entity interaction rates for one or more Operating Systems (OS), observed entity interaction rates for one or more campaign formats, or observed entity interaction rates for a particular campaign rendered on a particular third-party publisher systems.

12. The one or more non-transitory storage media of claim 9 , wherein generating the score is based at least in part on:
identifying interaction activities associated with the identified entity identifier;
determining that the inter

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Haribhaskaran Alagu Sanjana
• Shariat Talkhoonche Shahriar
• Wang Zhen
• Ma Yanbo

Classifications

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N20/10

  using kernel methods, e.g. support vector machines [SVM] · CPC title

• G06Q30/0243 Primary

  Comparative campaigns · CPC title

• G06N20/20

  Ensemble learning · CPC title