Lift reporting system

Patent metadata

Field	Value
Publication number	US-12361446-B2
Application number	US-202318328492-A
Country	US
Kind code	B2
Filing date	Jun 2, 2023
Priority date	Jun 2, 2023
Publication date	Jul 15, 2025
Grant date	Jul 15, 2025

Abstract

A lift reporting system to perform operations that include: accessing user behavior data associated with one or more machine-learned (ML) models, the ML models associated with identifiers; determining causal conversions associated with the ML models based on the user behavior data, the causal conversions comprising values; performing a comparison between the values that represents the causal conversions; determining a ranking of the ML models based on the comparison; and causing display of a graphical user interface (GUI) that includes a display of identifiers associated with ML models.

First claim

What is claimed is:

1. A system comprising:
at least one processor;
at least one memory component storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
causing display of a menu element at a client device, the menu element including a display of a set of identifiers that include a first identifier that identifies a first machine-learned (ML) model, and a second identifier that identifies a second ML model;
receiving tactile inputs at the client device that select the first identifier and the second identifier;
accessing first user behavior data associated with the first ML model and second user behavior data associated with the second ML model from within a database responsive to the inputs that select the first identifier and the second identifier;
determining a first set of causal conversions associated with the first ML model based on the first user behavior data, the first set of causal conversions comprising a first value;
determining a second set of causal conversions associated with the second ML model based on the second user behavior data, the second set of causal conversions comprising a second value;
performing a comparison between the first value that represents the first set of causal conversions and the second value that represents the second set of causal conversions;
determining a ranking of the first ML model and the second ML model based on the comparison; and
selecting a highest ranked ML model based on the ranking; and
identifying a set of candidate users to include in a media distribution campaign based on the highest ranked ML model.

2. The system of claim 1 , wherein the display of the first identifier includes the first value and the display of the second identifier includes the second value.

3. The system of claim 1 , wherein the GUI includes a visualization of the first value and the second value, the visualization including a bar graph.

4. The system of claim 1 , wherein the determining the first set of causal conversions associated with the first ML model and the second set of causal conversions associated with the second ML model includes:
accessing a matching service that identifies conversion events based on the first user behavior data and the second user behavior data; and
determining the first set of causal conversions and the second set of causal conversions based on the matching service.

5. The system of claim 1 , wherein the first ML model corresponds with a campaign, the second ML model comprises an update to the first ML model, and the method further comprises:
determining the second value that represents the second set of causal conversions is greater than the first value that represents the first set of causal conversions; and
applying the second ML model to the campaign based on the determining the second value is greater than the first value.

6. The system of claim 1 , wherein the performing the comparison between the first value that represents the first set of causal conversions and the second value that represents the second set of causal conversions includes:
performing an A/B test based on the first user behavior data and the second user behavior data.

7. A method comprising:
causing display of a menu element at a client device, the menu element including a display of a set of identifiers that include a first identifier that identifies a first machine-learned (ML) model, and a second identifier that identifies a second ML model;
receiving tactile inputs at the client device that select the first identifier and the second identifier;
accessing first user behavior data associated with the first ML model and second user behavior data associated with the second ML model from within a database responsive to the inputs that select the first identifier and the second identifier;
determining a first set of causal conversions associated with the first ML model based on the first user behavior data, the first set of causal conversions comprising a first value;
determining a second set of causal conversions associated with the second ML model based on the second user behavior data, the second set of causal conversions comprising a second value;
performing a comparison between the first value that represents the first set of causal conversions and the second value that represents the second set of causal conversions;
determining a ranking of the first ML model and the second ML model based on the comparison; and
selecting a highest ranked ML model based on the ranking; and
identifying a set of candidate users to include in a media distribution campaign based on the highest ranked ML model.

8. The method of claim 7 , wherein the display of the first identifier includes the first value and the display of the second identifier includes the second value.

9. The method of claim 7 , wherein the GUI includes a visualization of the first value and the second value, the visualization including a bar graph.

10. The method of claim 7 , wherein the determining the first set of causal conversions associated with the first ML model and the second set of causal conversions associated with the second ML model includes:
accessing a matching service that identifies conversion events based on the first user behavior data and the second user behavior data; and
determining the first set of causal conversions and the second set of causal conversions based on the matching service.

11. The method of claim 7 , wherein the first ML model corresponds with a campaign, the second ML model comprises an update to the first ML model, and the method further comprises:
determining the second value that represents the second set of causal conversions is greater than the first value that represents the first set of causal conversions; and
applying the second ML model to the campaign based on the determining the second value is greater than the first value.

12. The method of claim 8 , wherein the performing the comparison between the first value that represents the first set of causal conversions and the second value that represents the second set of causal conversions includes:
performing an A/B test based on the first user behavior data and the second user behavior data.

13. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
causing display of a menu element at a client device, the menu element including a display of a set of identifiers that include a first identifier that identifies a first machine-learned (ML) model, and a second identifier that identifies a second ML model;
receiving tactile inputs at the client device that select the first identifier and the second identifier;
accessing first user behavior data associated with the first ML model and second user behavior data associated with the second ML model from within a database responsive to the inputs that select the first identifier and the second identifier;
determining a first set of causal conversions associated with the first ML model based on the first user behavior data, the first set of causal conversions comprising a first value;
determining a second set of causal conversions associated with the second ML model based on the second user behavior data, the second set of causal conversions comprising a second value;
performing a comparison between the first value that represents the first set of causal conversions and the second value that represents the second set of causal conversions;
determining a ranking of the first ML mo

Assignees

• Snap Inc

Inventors

• Hollands Simon
• Shi Li
• Adur Amit
• Fumagalli Mattia
• Pan Crystal

Classifications

• G06Q30/0243 Primary

  Comparative campaigns · CPC title

• G06N20/00

  Machine learning · CPC title

• G06Q30/02011

  Profiling or inferring profiles of users or market based on their behaviour · CPC title