Geographic dataset preparation and analytics systems

Patent metadata

Field	Value
Publication number	US-12248957-B2
Application number	US-202217899202-A
Country	US
Kind code	B2
Filing date	Aug 30, 2022
Priority date	Mar 30, 2020
Publication date	Mar 11, 2025
Grant date	Mar 11, 2025

Abstract

Techniques for preparing datasets for geo experiments and improving accuracy of geo experiments are presented herein. The system can access a dataset of a plurality of geographic pairs. Additionally, the system can calculate a first outcome estimate based on a difference in response data and a difference in input data for a first geographic pair. Moreover, the system can calculate a plurality of experimental uncertainty estimates associated with the plurality of geographic pairs during an experimental time interval. The system can access historical data associated with the plurality of geographic pairs. Furthermore, the system can determine a beta value and a trim rate that reduces a sum of the plurality estimates. Subsequently, the system can remove, based on the first outcome estimate and the beta value, the first geographic pair from the plurality of geographic pairs to generate the first subset of geographic pairs.

First claim

What is claimed is:

1. A computer-implemented method for preparing datasets for geo experiments, the method comprising:
accessing, by the one or more computing devices, a dataset of a plurality of geographic pairs, the dataset of the plurality of geographic pairs comprising input data, response data, and location identifiers associated with each geographic region, wherein the response data is a result of an action associated with the input data, and wherein a first geographic pair of the dataset of the plurality of geographic pairs comprises a first geographic region and a second geographic region;
calculating, by the one or more computing devices, a first outcome estimate based on a difference in response data and a difference in input data for the first geographic pair;
calculating, by the one or more computing devices, a plurality of experimental uncertainty estimates associated with the plurality of geographic pairs during an experimental time interval, the plurality of experimental uncertainty estimates being calculated based on a plurality of different simulation for the plurality of geographic pairs during the experimental time interval;
accessing, by the one or more computing devices, historical data associated with the plurality of geographic pairs, the historical data being associated with a first time interval, the first time interval occurring prior to the experimental time interval, and wherein the historical data comprises of a historical response difference between the plurality of geographic pairs during the first time interval;
determining a beta value associated with the historical response difference between the plurality of geographic pairs during the first time interval, the beta value being determined to reduce a sum of the plurality of experimental uncertainty estimates of associated with the plurality of geographic pairs;
removing, based on the first outcome estimate and the beta value, the first geographic pair from the plurality of geographic pairs to generate a first subset of geographic pairs; and
providing, by the one or more computing devices, the first subset of geographic pairs.

2. The computer-implemented method of claim 1 , further comprising:
receiving an input parameter associated with a geo experiment for an entity;
calculating, using the first subset of geographic pairs and the input parameter, an incremental response on incremental input estimate; and
presenting, on a display of a content provider device, the incremental response on incremental input estimate.

3. The computer-implemented method of claim 2 , wherein the first geographic pair is removed from the plurality of geographic pairs further based on the input parameter, the method further comprising:
sending, based on the first subset of geographic pairs and the input parameter, a notification to the content provider device to initiate a content provider initiative.

4. The computer-implemented method of claim 2 , wherein the input parameter is the experimental time interval, and wherein the beta value is determined based on the experimental time interval.

5. The computer-implemented method of claim 2 , wherein the input parameter is a desired change in input level, and wherein the beta value is determined based on the desired change in input level.

6. The computer-implemented method of claim 2 , wherein the input parameter is a geographic area to target, and wherein the beta value is determined based on geographic area to target.

7. The computer-implemented method of claim 1 , wherein the beta value is further determined based on a confidence interval of a distribution curve associated with the plurality of experimental uncertainty estimates being below a certain threshold.

8. The computer-implemented method of claim 1 , wherein the dataset of the plurality of geographic pairs further comprises difference in response data during the experimental time interval, difference in input data during the experimental time interval, and difference in input data during the first time interval.

9. The computer-implemented method of claim 1 , wherein the historical data comprising historical input data and historical response data associated with each geographic, and wherein the historical response difference is calculated based on a difference in historical response data and a difference in historical input data.

10. The computer-implemented method of claim 1 , further comprising:
determining, by the one or more computing devices, a trim rate, the trim rate being determined to reduce a sum of the plurality of experimental uncertainty estimates associated with the plurality of geographic pairs; and
wherein the first geographic pair is removed, based on the trim rate, from the plurality of geographic pairs to generate the first subset of geographic pairs.

11. The computer-implemented method of claim 10 , wherein the trim rate is further determined based on a confidence interval of a distribution curve associated with the plurality of experimental uncertainty estimates being below a certain threshold.

12. The computer-implemented method of claim 1 , further comprising:
accessing data corresponding to a plurality of geographic regions, the data comprising input data, response data, and location identifiers associated with each geographic region;
calculating a difference in input data and a difference in response data for each geographic region of the plurality of geographic regions; and
determining the plurality of geographic pairs based on the difference in response data and the difference in input data for each geographic region of the plurality of geographic regions.

13. The computer-implemented method of claim 1 , wherein the plurality of different simulation for the plurality of geographic pairs are generated by separating the geographic region in a geographic pair of the plurality of geographic pairs into a treatment region or a control region for a plurality of simulations, wherein each simulation in the plurality of simulations generates an outcome estimate.

14. The computer-implemented method of claim 13 , wherein the plurality of experimental uncertainty estimates is further calculated based on the outcome estimate for each simulation in the plurality of simulations.

15. The computer-implemented method of claim 1 , wherein the response data is a key performance indicator, and the beta value is determined based on the key performance indicator.

16. A computing system, comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store:
a machine-learned model, wherein the machine-learned model is configured to generate a first subset of geographic pairs from a plurality of geographic pairs; and
instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
accessing a dataset of a plurality of geographic pairs, the dataset of the plurality of geographic pairs comprising input data, response data, and location identifiers associated with each geographic region, wherein the response data is a result of an action associated with the input data, and wherein a first geographic pair of the dataset of the plurality of geographic pairs comprises a first geographic region and a second geographic region;
calculating a first outcome estimate based on a difference in response data and a difference in input data for the first geographic pair;
calculating a plurality of experimental uncertainty estimates associated with the plurality of geographic pairs during an experimental time

Assignees

• Google Llc

Inventors

• Chen Aiyou
• Au Timothy Chun-Wai

Classifications

• G06Q30/0201

  Market modelling; Market analysis; Collecting market data · CPC title

• G06Q30/0241

  Advertisements · CPC title

• G06Q30/0205 Primary

  based on location or geographical consideration · CPC title