Simulation-based evaluation of a marketing channel attribution model

Patent metadata

Field	Value
Publication number	US-10387909-B2
Application number	US-201615005205-A
Country	US
Kind code	B2
Filing date	Jan 25, 2016
Priority date	Jan 25, 2016
Publication date	Aug 20, 2019
Grant date	Aug 20, 2019

Abstract

Techniques for managing a marketing campaign of a marketer are described. In an example, the marketing campaign uses multiple marketing channels. Attribution of each marketing channel to a user conversion is estimated. Usage of a marketing channel within the marketing campaign is set according to the respective attribution. A marketing channel attribution model is selected from candidate marketing channel attribution models and is applied to estimate the attributions. The selection is based on the accuracy of each of the models associated with estimating the attributions given a set of parameters. To evaluate the accuracy, user journeys are simulated given the set of parameters. True attributions of each marketing channel are determined from the simulation. Each of the marketing channel attribution models is also applied to the simulation to generate estimated attributions. The true and estimated attributions are compared to derive the accuracies.

First claim

The invention claimed is:

1. A method for allocating content-delivery resources among electronic content-delivery channels based on simulations of user behavior, wherein the method includes a simulation computing system performing one or more operations comprising:
establishing a communication session over a data network between the simulation computing system and a content management device remotely located from the simulation computing system;
during the communication session:
providing, to the content management device, an interface configured for inputting a set of parameters associated with simulating the user behavior with respect to the electronic content-delivery channels;
receiving, via the interface, the set of parameters from the content management device;
executing a simulation that comprises:
simulating user exposures as a Poisson random variable,
simulating one or more of (i) times between user exposures as an exponential random variable and (ii) electronic content-delivery channel use frequencies as a multinomial distribution, and
computing, based on the simulated user exposures and the one or more of (i) the simulated times between user exposures and (ii) the simulated electronic content-delivery channel use frequencies, simulated user responses resulting from the electronic content-delivery channels,

generating, based on the simulated user responses, model-selection data indicating an accuracy of an electronic channel attribution model when attributing a user response to an electronic content-delivery channel; and

causing a content delivery device to allocate the content-delivery resources to the electronic content-delivery channel in accordance with the model-selection data and thereby provide interactive content via the electronic content-delivery channel.

2. The method of claim 1 , wherein causing the content delivery device to allocate the content-delivery resources to the electronic content-delivery channel in accordance with the model-selection data comprises:
receiving, from the content management device, a request to estimate an attribution of the electronic content-delivery channel based on the accuracy of the electronic channel attribution model;
estimating the attribution of the electronic content-delivery channel by applying the electronic channel attribution model to the simulated user exposures and the simulated user responses; and
adjusting usage of the electronic content-delivery channel by the content delivery device based on the estimated attribution of the electronic content-delivery channel.

3. The method of claim 1 , wherein causing the content delivery device to allocate the content-delivery resources to the electronic content-delivery channel in accordance with the model-selection data comprises:
receiving, from the content management device, a request to use a different electronic channel attribution model based on the accuracy of the electronic channel attribution model; and
estimating an attribution of the electronic content-delivery channel based on the different electronic channel attribution model; and
adjusting usage of the electronic content-delivery channel by the content delivery device based on the estimated attribution of the electronic content-delivery channel.

4. The method of claim 1 , further comprising:
receiving, from the content management device, a request to evaluate the electronic channel attribution model and a different electronic channel attribution model;
evaluating an accuracy of the different electronic channel attribution model based on the simulated user exposures and the simulated user responses;
ranking the electronic channel attribution model and the different electronic channel attribution model based on the accuracy of the electronic channel attribution model and based on the accuracy of the different electronic channel attribution model;
providing the ranking of the electronic channel attribution model and the different electronic channel attribution model to the content management device;
receiving, from the content management device, a request to use the electronic channel attribution model for estimating an attribution of the electronic content-delivery channel based on the ranking; and
estimating the attribution of the electronic content-delivery channel based on the electronic channel attribution model.

5. The method of claim 1 , further comprising:
receiving, from the content management device, input identifying a different electronic channel attribution model, a selection rule that specifies a selection between the electronic channel attribution model and the different electronic channel attribution model, and an adjustment rule that specifies an adjustment to the electronic content-delivery channel;
ranking the electronic channel attribution model and the second marketing different electronic channel attribution model based on the simulated user exposures and the simulated user responses;
selecting the electronic channel attribution model based on the ranking and based on the selection rule;
estimating an attribution of the electronic content-delivery channel based on the electronic channel attribution model; and
adjusting usage of the electronic content-delivery channel by the content delivery device based on the estimated attribution and the adjustment rule.

6. The method of claim 1 , wherein executing the simulation comprises simulating user behaviors, user transitions through conversion stages, electronic content-delivery channel exposures, effects of combinations of electronic content-delivery channel exposures, and decays of the electronic content-delivery channel exposures.

7. The method of claim 1 , wherein evaluating the electronic channel attribution model is comprises:
generating true attributions corresponding to the electronic content-delivery channels based on the simulated user exposures and the simulated user responses;
generating estimated attributions corresponding to the electronic content-delivery channels, the estimated attributions generated based on the electronic channel attribution model;
evaluating the accuracy of the electronic channel attribution model based on a comparison of the true attributions to corresponding estimated attributions.

8. The method of claim 1 , further comprising:
removing a subset of the simulated user exposures corresponding to exposures to the electronic content-delivery channel;
determining a subset of the simulated user responses corresponding to the subset of the simulated user exposures; and
generating a true attribution of the electronic content-delivery channel based on a comparison of the simulated user responses to the subset of the simulated user responses.

9. The method of claim 1 , further comprising:
estimating an attribution of the electronic content-delivery channel based on the electronic channel attribution model;
generating a first metric for estimating the accuracy of the electronic channel attribution model, the first metric generated based on a true attribution of the electronic content-delivery channel and the estimated attribution of the electronic content-delivery channel; and
generating a second metric for comparing the accuracy of the electronic channel attribution model to another accuracy of another electronic channel attribution model,
wherein the model-selection data comprises the first metric and the second metric.

10. The method of claim 1 , further comprising providing the model-selection data to the content management device via the interface.

11. A system comprising:
a content delivery device; and
a simulation computing device communicatively coupled to the content delivery dev

Assignees

• Adobe Inc

Inventors

• Yadagiri Meghanath Macha
• Sinha Ritwik
• Saini Shiv Kumar

Classifications

• G06Q30/0244 Primary

  Optimization · CPC title

• G06Q10/067

  Enterprise or organisation modelling · CPC title