Mitigating bias in lift analysis for demand side platforms

Patent metadata

Field	Value
Publication number	US-2017061467-A1
Application number	US-201514839879-A
Country	US
Kind code	A1
Filing date	Aug 28, 2015
Priority date	Aug 28, 2015
Publication date	Mar 2, 2017
Grant date	—

Abstract

A demand side platform (DSP) performs lift analysis on one or more advertisements provided by an entity to determine how effective the one or more advertisements cause users to perform various actions. To reduce bias in the lift analysis, when the DSP identifies an opportunity to present an advertisement to a user, the DSP identifies candidate advertisements for presentation to the user and selects a candidate advertisement. If the candidate advertisement is undergoing lift analysis and the user is in a holdout group that is not presented with the candidate advertisement, the DSP selects an alternative advertisement and associates a bid amount for the selected advertisement with the alternative advertisement. If the alternative advertisement is presented to the user, the DSP may allocate an amount charged between an entity associated with the alternative advertisement and an entity associated with the selected advertisement.

First claim

What is claimed is:

1 . A method comprising:
maintaining a plurality of advertisements, each advertisement including targeting criteria identifying characteristics of users eligible to be presented with an advertisement and bid information specifying an amount of compensation from an entity associated with the advertisement in exchange for presentation of the advertisement; receiving a request for one or more advertisements for presentation to a user from an online system, the request including information associated with the user to be presented with the one or more advertisements; selecting one or more candidate advertisements from the maintained plurality advertisements, each candidate advertisement including at least a threshold number of targeting criteria satisfied by the information associated with the user; computing bid amounts for each of the candidate advertisements, a bid amount for a candidate advertisement based at least in part on bid information associated with the candidate advertisement and targeting criteria included in the candidate advertisement satisfied by the information associated with the user; selecting a candidate advertisement based at least in part on the computed bid amounts; determining that the selected candidate advertisement is undergoing lift analysis to determine an effectiveness of the selected candidate advertisement; determining that the user is part of a holdout group for the selected candidate advertisement, the holdout group identifying users to whom the selected candidate advertisement is not shown; identifying an alternative candidate advertisement based at least in part on the computed bid amounts; and communicating a response to the online system including the alternative candidate advertisement in association with the bid amount for the selected candidate advertisement to the online system.

2 . The method of claim 1 , wherein the selected candidate advertisement comprises a candidate advertisement having a highest bid amount, and wherein identifying the alternative candidate advertisement based at least in part on the computed bid amounts comprises:
identifying the alternative candidate advertisement as a candidate advertisement having a second highest bid amount.

3 . The method of claim 1 , further comprising:
determining a difference between the bid amount of the selected candidate advertisement and a bid amount of the alternative candidate advertisement in response to receiving an indication the alternative candidate advertisement was selected for presentation to the user; charging the bid amount of the alternative candidate advertisement to an entity associated with the alternative candidate advertisement; and charging an entity associated with the selected candidate advertisement the determined difference.

4 . The method of claim 1 , further comprising:
charging the bid amount of the selected candidate advertisement in part to an entity associated with the selected candidate advertisement and in part to an entity associated with the alternative candidate advertisement in response to receiving an indication the alternative candidate advertisement was selected for presentation to the user.

5 . The method of claim 1 , further comprising:
charging the bid amount of the alternative candidate advertisement to an entity associated with the alternative candidate advertisement in response to receiving an indication the alternative candidate advertisement was selected for presentation to the user.

6 . The method of claim 1 , wherein identifying the alternative candidate advertisement based at least in part on the computed bid amounts comprises:
determining the alternative candidate advertisement is undergoing lift analysis to determine an effectiveness of the alternative candidate advertisement; determining that the user is part of a holdout group for the alternative candidate advertisement, the holdout group identifying users to whom the alternative candidate advertisement is not shown; and identifying an additional candidate advertisement based at least in part on the determined bid amounts.

7 . The method of claim 6 , wherein communicating the response to the online system including the alternative candidate advertisement in association with the bid amount for the selected candidate advertisement to the online system comprises:
communicating a response to the online system including the additional candidate advertisement in association with the bid amount for the selected candidate advertisement to the online system.

8 . The method of claim 6 , wherein identifying the additional candidate advertisement based at least in part on the determined bid amounts comprises:
identifying a candidate advertisement having a bid amount lower than a bid amount for the alternative candidate advertisement as the additional candidate advertisement.

9 . The method of claim 1 , wherein the alternative candidate advertisement includes a tracking mechanism that, when executed by a client device presenting the alternative candidate advertisement, communicates information describing one or more interactions with the alternative candidate advertisement from the client device and identifying the selected candidate advertisement.

10 . The method of claim 1 , wherein an entity associated with the selected candidate advertisement identifies a size of the holdout group for the selected candidate advertisement.

11 . A computer program product comprising a computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:
maintain a plurality of advertisements, each advertisement including targeting criteria identifying characteristics of users eligible to be presented with an advertisement and bid information specifying an amount of compensation from an entity associated with the advertisement in exchange for presentation of the advertisement; receive a request for one or more advertisements for presentation to a user from an online system, the request including information associated with the user to be presented with the one or more advertisements; select one or more candidate advertisements from the maintained plurality advertisements, each candidate advertisement including at least a threshold number of targeting criteria satisfied by the information associated with the user; compute bid amounts for each of the candidate advertisements, a bid amount for a candidate advertisement based at least in part on bid information associated with the candidate advertisement and targeting criteria included in the candidate advertisement satisfied by the information associated with the user; select a candidate advertisement based at least in part on the computed bid amounts; determine that the selected candidate advertisement is undergoing lift analysis to determine an effectiveness of the selected candidate advertisement; determine that the user is part of a holdout group for the selected candidate advertisement, the holdout group identifying users to whom the selected candidate advertisement is not shown; identify an alternative candidate advertisement based at least in part on the computed bid amounts; and communicate a response to the online system including the alternative candidate advertisement in association with the bid amount for the selected candidate advertisement to the online system.

12 . The computer program product of claim 11 , wherein the selected candidate advertisement comprises a candidate advertisement having a highest bid amount, and wherein identify the alternative candidate advertisement based at least in part on the computed bi

Assignees

• Facebook Inc

Inventors

• Mchugh Jason George
• Litvak Emile Vladimir

Classifications

• G06Q30/0242 Primary

  Determining effectiveness of advertisements · CPC title

• G06Q30/0269

  based on user profile or attribute · CPC title

• G06Q30/0277

  Online advertisement · CPC title

• G06Q30/0251 Primary

  Targeted advertisements · CPC title

• G06Q30/0244

  Optimization · CPC title