Generation of models for classifying user groups

Patent metadata

Field	Value
Publication number	US-12260422-B2
Application number	US-202217655724-A
Country	US
Kind code	B2
Filing date	Mar 21, 2022
Priority date	Mar 21, 2022
Publication date	Mar 25, 2025
Grant date	Mar 25, 2025

Abstract

Methods, systems, and computer programs are presented for estimating if a user belongs to an audience category. One method includes an operation for accessing events generated at a website. Each event comprises a data structure describing an operation performed by a user, from a group of users, when accessing the website. Further, the method includes an operation for providing event information and information of a first user, for a predefined time window, as input to an audience machine-learning (ML) model. The audience ML model is trained with training data comprising values for features that include event features, user information features, and audience labels. The method further includes operations for generating, by the audience ML model, a score for the first user indicating a probability that the first user belongs to the audience, and for determining if the user belongs to the audience based on the score.

First claim

What is claimed is:

1. A computer-implemented method comprising:
generating, by one or more processors, training data that comprises values for features that include event features, user information features, audience labels, and product features, the generating of the training data including labelling each event among events within a first time window, the labelling of the events distinguishing first events associated with an audience category from second events not associated with the audience category;
training, by the one or more processors, a propensity-to-buy machine-learning (ML) model based on the generated training data that comprises the labelled events and the values for the features that include the event features, the user information features, the audience labels, and the product features;
accessing, by the one or more processors, further events, each further event comprising a data structure describing an operation performed by a single persona identified among a plurality of users;
providing, by the one or more processors, event information of the further events and information of the single persona as input to the propensity-to-buy ML model trained based on the training data that comprises the labelled events and the values for the features that include the event features, the user information features, the audience labels, and the product features;
generating, by the propensity-to-buy ML model, a score for the single persona indicating a probability that the single persona belongs to the audience category and will purchase a product within a future time window;
determining, by the one or more processors, a total value of billing events for the single persona over the future time window based on the score;
updating, by the one or more processors, the training data based on at least one of a new event feature or an obsolete event feature, the updating of the training data including labelling each event among events within a second time window, the labelling of the events distinguishing third events associated with the audience category from fourth events not associated with the audience category; and
retraining, by the one or more processors, the propensity-to-buy ML model based on the updated training data.

2. The method as recited in claim 1 , wherein the event features are selected from a group consisting of a number of orders in a feature window (FW), a number of items ordered in the FW, a number of items added to a cart in the FW, a number of page view in the FW, a number of cart views in the FW, a number of days since a last order, and a total value of purchases in the FW.

3. The method as recited in claim 1 , wherein the user information features are selected from a group consisting of an email of the single persona, a telephone number of the single persona, and an internet protocol (IP) address of a device utilized by the single persona.

4. The method as recited in claim 1 , further comprising:
calculating the audience labels in the training data based on a predefined rule.

5. The method as recited in claim 1 , wherein the further events are selected from a group consisting of accessing a product webpage, ordering a product, adding a product to an electronic shopping cart, adding a product to a user wish list, and viewing the electronic shopping cart.

6. The method as recited in claim 1 , further comprising:
analyzing the events within the first time window; and
determining a persona identifier for each analyzed event, each persona identifier being associated with a user with one or more user traits, the user traits comprising at least one of an email address, an IP address, and a telephone number.

7. The method as recited in claim 1 , further comprising:
generating a forecast of purchases of the product for the plurality of users.

8. A system comprising:
a memory comprising instructions; and
one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising:
generating training data that comprises values for features that include event features, user information features, audience labels, and product features, the generating of the training data including labelling each event among events within a first time window, the labelling of the events distinguishing first events associated with an audience category from second events not associated with the audience category;
training a propensity-to-buy machine-learning (ML) model based on the generated training data that comprises the labelled events and the values for the features that include the event features, the user information features, the audience labels, and the product features;
accessing further events, each further event comprising a data structure describing an operation performed by a single persona identified among a plurality of users;
providing event information of the further events and information of the single persona as input to the propensity-to-buy ML model trained based on the training data that comprises the labelled events and the values for the features that include the event features, the user information features, the audience labels, and the product features;
generating, by the propensity-to-buy ML model, a score for the single persona indicating a probability that single persona belongs to the audience category and will purchase a product within a future time window;
determining a total value of billing events for the single persona over the future time window based on the score;
updating the training data based on at least one of a new event feature or an obsolete event feature, the updating of the training data including labelling each event among events within a second time window, the labelling of the events distinguishing third events associated with the audience category from fourth events not associated with the audience category; and
retraining the propensity-to-buy ML model based on the updated training data.

9. The system as recited in claim 8 , wherein the event features are selected from a group consisting of a number of orders in a feature window (FW), a number of items ordered in the FW, a number of items added to a cart in the FW, a number of page view in the FW, a number of cart views in the FW, a number of days since a last order, and a total value of purchases in the FW.

10. The system as recited in claim 8 , wherein the user information features are selected from a group consisting of an email of the single persona, a telephone number of the single persona, and an internet protocol (IP) address of a device utilized by the single persona.

11. The system as recited in claim 8 , wherein the instructions further cause the one or more computer processors to perform operations comprising:
calculating the audience labels in the training data based on a predefined rule.

12. The system as recited in claim 8 , wherein the further events are selected from a group consisting of accessing a product webpage, ordering a product, adding a product to an electronic shopping cart, adding a product to a user wish list, and viewing the electronic shopping cart.

13. The system as recited in claim 8 , wherein the instructions further cause the one or more computer processors to perform operations comprising:
analyzing the events within the first time window; and
determining a persona identifier for each analyzed event, each persona identifier being associated with a user with one or more user traits, the user traits comprising at least one of an email address, an IP address, and a telephone number.

14. The system as recit

Assignees

• Twilio Inc

Inventors

• Woytarowicz Nicole
• Tucker Samuel Vaughn
• Mgherbrishvili Inga
• Wing Rowan Michael
• Wang Hanhan

Classifications

• G06Q30/0202

  Market predictions or forecasting for commercial activities · CPC title

• G06N20/00

  Machine learning · CPC title

• G06Q30/0204 Primary

  Market segmentation · CPC title