Geographic dataset preparation system

Patent metadata

Field	Value
Publication number	US-11514274-B2
Application number	US-202016834843-A
Country	US
Kind code	B2
Filing date	Mar 30, 2020
Priority date	Mar 30, 2020
Publication date	Nov 29, 2022
Grant date	Nov 29, 2022

Abstract

Systems, methods and computer-readable storage media utilized to prepare datasets for geo experiments. One method includes receiving one or more input parameters. The method further includes extracting, from the data, training data. The method further includes calculating a difference in input data and a difference in response data of the training data. The method further includes determining a first plurality of geographic pairs. The method further includes extracting, from the data, evaluation data. The method further includes separating each geographic pair of the first plurality of geographic pairs into a treatment region or a control region for a plurality of simulations of a plurality of different simulation subsets for each of a plurality of different subsets of geographic pairs. The method further includes calculating a plurality of uncertainty estimates. The method further includes selecting a first subset of geographic pairs and providing the selected subset of geographic pairs.

First claim

What is claimed is:

1. A computer-implemented method of preparing datasets for geo experiments, comprising:
receiving, by one or more processing circuits, one or more input parameters associated with a geo experiment for an entity;
receiving, by the one or more processing circuits, data corresponding to a plurality of geographic regions, the data comprising input data, response data, and location identifiers associated with each geographic region, wherein the response data is a result of an action associated with the input data;
extracting, from the data corresponding to the plurality of geographic regions, by the one or more processing circuits, training data associated with a first time interval;
calculating, by the one or more processing circuits, a difference in input data and a difference in response data of the training data for each geographic region of the plurality of geographic regions;
determining, by the one or more processing circuits, a first plurality of geographic pairs based on the difference in response data and the difference in input data of the training data for each geographic region of the plurality of geographic regions;
extracting, from the data corresponding to the plurality of geographic regions, evaluation data associated with a second time interval;
separating, by the one or more processing circuits, the geographic regions of each geographic pair of the first plurality of geographic pairs into a treatment region or a control region for a plurality of simulations of a plurality of different simulation subsets for each of a plurality of different subsets of geographic pairs, wherein each simulation generates an outcome estimate;
calculating, by the one or more processing circuits, a plurality of uncertainty estimates based on the plurality of different simulation subsets for each of the plurality of different subsets of geographic pairs and the one or more input parameters, wherein each uncertainty estimate comprises a different subset of geographic pairs, and wherein each subset of geographic pairs comprises a different number of geographic pairs;
selecting, by the one or more processing circuits, a first subset of geographic pairs of the plurality of different subsets of geographic pairs based on the uncertainty estimates; and
providing, by the one or more processing circuits, the selected subset of geographic pairs.

2. The method of claim 1 , wherein generating an outcome estimate is based on calculating the difference in response data and the difference in input data of the evaluation data.

3. The method of claim 1 , wherein calculating the plurality of uncertainty estimates further comprises calculating a root mean square error.

4. The method of claim 1 , wherein the treatment region for the geographic regions of each geographic pair is associated with a first geographic region, and wherein the control region for the geographic regions of each geographic pair is associated with a second geographic region.

5. The method of claim 4 , wherein the first geographic region associated with the treatment region and the second geographic region associated with the control region is randomly selected from the geographic regions of each geographic pair for each of the plurality of simulations.

6. The method of claim 4 , wherein the first geographic region and the second geographic region are associated with a target population.

7. The method of claim 1 , wherein the one or more input parameters comprises at least one of an experiment time interval, one or more geographic locations, a target estimate, and an input amount.

8. The method of claim 1 , wherein the second time interval is smaller than the first time interval and is based at least on the one or more input parameters.

9. The method of claim 1 , wherein the second time interval is in the first time interval, and wherein the evaluation data comprises both a subset of the response data and a subset of input data of the training data.

10. A system comprising:
at least one processing circuit configured to:
receive one or more input parameters associated with a geo experiment for an entity;
receive data corresponding to a plurality of geographic regions, the data comprising input data, response data, and location identifiers associated with each geographic region, wherein the response data is a result of an action associated with the input data;
extract, from the data corresponding to a plurality of geographic regions, training data associated with a first time interval;
calculate a difference in input data and a difference in response data of the training data for each geographic region of the plurality of geographic regions;
determine a first plurality of geographic pairs based on the difference in response data and the difference in input data of the training data for each geographic region of the plurality of geographic regions;
extract, from the data corresponding to a plurality of geographic regions, evaluation data associated with a second time interval;
separate the geographic regions of each geographic pair of the first plurality of geographic pairs into a treatment region or a control region for a plurality of simulations of a plurality of different simulation subsets for each of a plurality of different subsets of geographic pairs, wherein each simulation generates an outcome estimate;
calculate a plurality of uncertainty estimates based on the plurality of different simulation subsets for each of the plurality of different subsets of geographic pairs and the one or more input parameters, wherein each uncertainty estimate comprises a different subset of geographic pairs, and wherein each subset of geographic pairs comprises a different number of geographic pairs;
select a first subset of geographic pairs of the plurality of different subsets of geographic pairs based on the uncertainty estimates; and
provide the selected subset of geographic pairs.

11. The system of claim 10 , wherein generating an outcome estimate is based on calculating the difference in response data and the difference in input data of the evaluation data.

12. The system of claim 10 , wherein calculating the plurality of uncertainty estimates further comprises calculating a root mean square error.

13. The system of claim 10 , wherein the treatment region for the geographic regions of each geographic pair is associated with a first geographic region, and wherein the control region for the geographic regions of each geographic pair is associated with a second geographic region.

14. The system of claim 13 , wherein the first geographic region associated with the treatment region and the second geographic region associated with the control region is randomly selected from the geographic regions of each geographic pair for each of the plurality of simulations.

15. The system of claim 13 , wherein the first geographic region and the second geographic region are associated with a target population.

16. The system of claim 10 , wherein the one or more input parameters comprises at least one of an experiment time interval, one or more geographic locations, a target estimate, and an input amount.

17. The system of claim 10 , wherein the second time interval is smaller than the first time interval and is based at least on the one or more input parameters.

18. One or more non-transitory computer-readable storage media having instructions stored thereon that, when executed by at least one processing circuit, cause the at least one processing circuit to perform operations comprising:
receiving one or m

Assignees

• Google Llc

Inventors

• Chen Aiyou
• Au Timothy C
• Remy Nicolas
• Benac Kevin

Classifications

• G06Q30/0241 Primary

  Advertisements · CPC title

• G06F18/2413

  based on distances to training or reference patterns · CPC title

• G06F18/2115

  by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title