Smart measurement points

Patent metadata

Field	Value
Publication number	US-11023958-B2
Application number	US-201815883008-A
Country	US
Kind code	B2
Filing date	Jan 29, 2018
Priority date	Jan 29, 2018
Publication date	Jun 1, 2021
Grant date	Jun 1, 2021

Abstract

Systems and methods for dynamically defining smart measurement points are disclosed. A first experiment configured to provide one of a base element or a variant element is defined and the first experiment is associated with a first page. The first experiment includes a first dynamically defined qualifying measurement point. A plurality of records each comprising one or more variables indicative of an interaction between one of the plurality of user systems and the computing device are generated and a search query including the first dynamically defined qualifying measurement point is received. Each record received is searched. For each record containing the first dynamically defined qualifying measurement point, an associated one of the plurality of user devices is qualified for the at least one experiment and, for each remaining record, the associated one of the plurality of user devices is excluded for the at least one experiment.

First claim

What is claimed is:

1. A system, comprising:
a computing device communicatively coupled to a database, where the computing device is configured to:
receive a first experiment configured to provide each user system of a plurality of user systems one of a base element or a variant element, wherein the first experiment is associated with a first page, and wherein the first experiment includes a first dynamically defined qualifying measurement point indicating a qualifying event corresponding to one or more user interactions of a corresponding user system of the plurality of user systems with the first page;
generate a plurality of records based on user interaction data stored within the database, each record comprising one or more variables indicative of an interaction between one of the plurality of user systems and the computing device;
receive a search query including the first dynamically defined qualifying measurement point;
search each record of the plurality of records received at the computing device, wherein:
for each record containing the first dynamically defined qualifying measurement point, qualify the corresponding one of the plurality of user systems for the first experiment; and
for each remaining record, exclude the corresponding one of the plurality of user systems for the first experiment;

analyze the variant element based on records corresponding to qualified user systems of the plurality of user systems; and
generate one or more outputs based on the analysis of the variant element.

2. The system of claim 1 , wherein the first experiment includes at least one reportable measurement point, and wherein the computing device is further configured to compare the qualified user systems to the at least one reportable measurement points.

3. The system of claim 1 , wherein the first experiment includes at least one system defined measurement point, wherein each record containing the first dynamically defined qualifying measurement point is qualified for the first experiment only when the record includes the at least one system defined measurement point.

4. The system of claim 1 , wherein the computing device is further configured to receive a second dynamically defined measurement point.

5. The system of claim 4 , wherein the second dynamically defined measurement point is defined after at least one user system has qualified for the first experiment.

6. The system of claim 4 , wherein, for each record containing the second dynamically defined measurement point, the computing device is configured to exclude the corresponding one of the plurality of user systems from the first experiment.

7. The system of claim 1 , wherein the computing device is further configured to:
review a syntax of the first dynamically defined measurement point; and
generate an alert when the syntax of the first dynamically defined measurement point fails to qualify any user system.

8. The system of claim 1 , wherein the computing device is further configured to:
define a second experiment configured to provide one of a second base element or a second variant element, wherein the second experiment is associated with a second page, and wherein the second experiment includes a second dynamically defined qualifying measurement point;
receive a search query including the second dynamically defined qualifying measurement point;
search each record received at the computer system, wherein:
for each record containing the second dynamically defined qualifying measurement point, qualify corresponding one of the plurality of user systems for the second experiment, wherein the second dynamically defined qualifying measurement point is different than the first dynamically defined qualifying measurement point.

9. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by a processor cause a device to perform operations comprising:
receiving a first experiment configured to provide each user system of a plurality of user systems one of a base element or a variant element, wherein the first experiment is associated with a first page, and wherein the first experiment includes a first dynamically defined qualifying measurement point indicating a qualifying event corresponding to one or more user interactions of a corresponding user system of the plurality of user systems with the first page;
generating a plurality of records based on user interaction data, each record comprising one or more variables indicative of an interaction between one of a plurality of user systems and the computing device;
receiving a search query including the first dynamically defined qualifying measurement point;
searching each record of the plurality of records, wherein:
for each record containing the first dynamically defined qualifying measurement point, qualify corresponding one of the plurality of user systems for the first experiment; and
for each remaining record, exclude the corresponding one of the plurality of user systems for the first experiment.

10. The non-transitory computer readable medium of claim 9 , wherein the first experiment includes at least one reportable measurement point, and wherein the method further comprises comparing the qualified user systems to the at least one reportable measurement points.

11. The non-transitory computer readable medium of claim 9 , wherein the first experiment includes at least one system defined measurement point, wherein each record containing the first dynamically defined qualifying measurement point is qualified for the first experiment only when the record includes the at least one system defined measurement point.

12. The non-transitory computer readable medium of claim 9 , comprising receiving a second dynamically defined measurement point.

13. The non-transitory computer readable medium of claim 12 , wherein the second dynamically defined measurement point is defined after at least one user system has qualified for the first experiment.

14. The non-transitory computer readable medium of claim 12 , wherein, for each record containing the second dynamically defined measurement point, the corresponding one of the plurality of user systems is excluded from the first experiment.

15. The non-transitory computer readable medium of claim 9 , wherein the computing device is further configured to:
review a syntax of the first dynamically defined measurement point; and
generate an alert when the syntax of the first dynamically defined measurement point fails to qualify any user system.

16. The non-transitory computer readable medium of claim 9 , wherein the instructions further cause the device to perform operations comprising:
defining a second experiment configured to provide one of a second base element or a second variant element, wherein the second experiment is associated with a second page, and wherein the second experiment includes a second dynamically defined qualifying measurement point;
receiving a search query including the second dynamically defined qualifying measurement point;
searching each record, wherein:
for each record containing the second dynamically defined qualifying measurement point, qualify corresponding one of the plurality of user systems for the first experiment, wherein the second dynamically defined qualifying measurement point is different than the first dynamically defined qualifying measurement point.

17. A method, comprising:
receiving a first experiment configured to provide each user system of a plurality of user systems one of a base element or a variant ele

Assignees

• Walmart Apollo Llc

Inventors

• Tang Anthony
• Kasireddi Swetharam
• Rubin Justin
• Bozchelouei Gholamreza Esfandani
• Subramanian Girish
• Le Duy
• Sharma Shriram
• Bitondo Stephen

Classifications

• G06Q30/0641 Primary

  utilising user interfaces specially adapted for shopping · CPC title

• G06F16/9038

  Presentation of query results · CPC title

• G06F8/42

  Syntactic analysis · CPC title

• G06Q30/0201

  Market modelling; Market analysis; Collecting market data · CPC title

• G06F9/451

  Execution arrangements for user interfaces · CPC title