Machine Learning for Predicting Incremental Changes in Session Data

Patent metadata

Field	Value
Publication number	US-2024242107-A1
Application number	US-202218009439-A
Country	US
Kind code	A1
Filing date	Jun 24, 2022
Priority date	Jun 24, 2022
Publication date	Jul 18, 2024
Grant date	—

Abstract

A computing system and method that can be used for facilitating data processing and system modeling of techniques used to transmit network resources over a communication network. In particular, a machine learning system can execute predictive models for predicting probabilities of user session data based on feature data. For example, the computing system can predict that a user may have a low likelihood of engaging in a particular action (e.g., downloading specified content, completing a transaction, interacting with a specific icon, widget, or application, launching a specific script, or some other specified action) prior to engaging with the network resource but have a high likelihood of engaging in the particular action post engaging with the network resource. In particular, the computing system can provide for generating an incremental label which indicates a probability of how likely a particular user is to change from not engaging in a particular action to engaging in a particular action after engaging with the network resource.

First claim

1 . A computer-implemented method, comprising:
providing, by a computing system comprising one or more processors, a network resource to a first plurality of users; obtaining, by the computing system, a first set of index data associated with the first plurality of users, the first set of index data describing first user session data for the first plurality of users subsequent to receipt of the network resource; obtaining, by the computing system, a second set of index data associated with a second plurality of users, the second set of index data describing second user session data for the second plurality of users in the absence of the network resource; training, by the computing system, one or more machine learning models based on the first set of index data and the second set of index data; generating, by the computing system using the one or more machine learning models, a first probability and a second probability for each of a third plurality of users based on feature data associated with such user, wherein the first probability is a respective probability of user session data subsequent to receipt of the network resource and the second probability is a respective probability of the user session data in absence of the network resource; generating, by the computing system, an incremental label for each of the third plurality of users, wherein the respective incremental label for each of the third plurality of users is descriptive of a difference in the first probability and the second probability for such user.

2 . The computer-implemented method of claim 1 , comprising:
ranking, by the computing system, the third plurality of users based at least in part on the incremental label; determining, by the computing system, whether to provide the network resource to each of the third plurality of users based at least in part on the ranking; and providing, by the computing system, the network resource to each of the third plurality of users for which a determination was made to provide the network resource.

3 . The computer-implemented method of claim 1 , wherein the one or more machine learning models comprise:
a first machine learning model trained on the first set of index data and configured to output a first prediction that describes a probability of user session data subsequent to receipt of the network resource; a second machine learning model trained on the second set of index data and configured to output a second prediction that describes a probability of user session data in the absence of the network resource; and wherein generating, by the computing system, the incremental label comprises determining a difference between the first prediction and the second prediction.

4 . The computer-implemented method of claim 3 , comprising:
generating a modified output of the first machine learning model, wherein the modified output comprises fitting the output of the first machine learning model, and wherein fitting the output comprises removing bias due to few data points; generating a modified output of the second machine learning model, wherein the modified output comprises fitting the output of the second machine learning model, and wherein fitting the output comprises removing bias due to few data points; leveraging a third machine learning model wherein the third machine learning model inputs the modified output of the first machine learning model; leveraging a fourth machine learning model wherein the fourth machine learning model inputs the modified output of the second machine learning model; generating a propensity score based on the outputs of the third and fourth machine learning models; and wherein generating, by the computing system, the incremental label comprises combining the outputs of the third and fourth machine learning model based on the propensity score.

5 . The computer-implemented method of claim 1 , wherein the one or more machine learning models comprise:
a single machine learning model trained on a combined set of data wherein the combined set of data includes the first set of index data and the second set of index data and wherein the single machine learning model is configured to output a first probability wherein the first probability is directed to a respective probability of user session data subsequent to receipt of the network resource and a second probability wherein the second probability is directed to a respective probability of user session data in absence of the network resource for each of a third plurality of users.

6 . The computer-implemented method of claim 1 , comprising:
generating, by the computing system, a third set of index data wherein the third set of index data comprises applying a first treatment value to the first set of index data and a second treatment value to the second set of index data; and training, by the computing system, the one or more machine learning models based on the third set of index data.

7 . The computer-implemented method of claim 1 , wherein the one or more machine learning models comprise:
a first machine learning model trained on a combined set of data wherein the combined set of data comprises the first set of index data and the second set of index data and wherein the first machine learning model is configured to output a prediction that describes whether a particular user received treatment or not; a second machine learning model trained on the combined set of data and configured to output a prediction that describes user session data; a modified set of data wherein the modified set of data comprises subtracting the output of the first machine learning model and the output of the second machine learning model from the combined set of data; and a third machine learning model trained on the modified set of data configured to output the incremental label.

8 . The computer-implemented method of claim 1 , comprising:
determining, by the computing system, a content value associated with the network resource, wherein the content value is a parameter quantifying a similarity in content between first and second network resources; wherein the first set of index data and the second set of index data is obtained in response to the first network resource; wherein the respective incremental label for each of the third plurality of users is descriptive of a predicted change in user session data effected by providing the second network resource to the user; and wherein generating the respective incremental label for each of the third plurality of users is based at least in part on a combination of feature data associated with the user and the content value.

9 . The computer-implemented method of claim 1 , comprising:
generating, by the computing system, a graphical illustration based at least in part on the incremental label, the first set of index data, and the second set of index data; and surfacing, by the computing system, the graphical illustration to a user.

10 . The computer-implemented method of claim 1 , wherein the incremental label is a first incremental label, the method, comprising:
generating, by the computing system using the one or more machine learning models, a second respective incremental label for each of the third plurality of users based on the feature data associated with the user.

11 . The computer-implemented method of claim 10 , comprising:
generating, by the computing system, a comparison score wherein the comparison score is a parameter descriptive of the difference between the first incremental label and the second incremental label.

12 . The computer-implemented method of claim 11 further comprising:
determining, by the comput

Assignees

• Google Llc

Inventors

• Sun Ming
• Chaisiri Teresa

Classifications

• G06N20/00 Primary

  Machine learning · CPC title

• G06N5/01 Primary

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title