Actionable KPI-driven segmentation

Patent metadata

Field	Value
Publication number	US-11651383-B2
Application number	US-201816191289-A
Country	US
Kind code	B2
Filing date	Nov 14, 2018
Priority date	Nov 14, 2018
Publication date	May 16, 2023
Grant date	May 16, 2023

Abstract

An improved analytics system generates actionable KPI-based customer segments. The analytics system determines predicted outcomes for a key performance indicator (KPI) of interest and a contribution value for each variable indicating an extent to which each variable contributes to predicted outcomes. Topics are generated by applying a topic model to the contribution values for the variables. Each topic comprises a group of variables with a contribution level for each variable that indicates the importance of each variable to the topic. User segments are generated by assigning each user to a topic based on attribution levels output by the topic model.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
converting, by one or more processor, raw event data to training data by performing feature engineering;
based the converting, training, by the one or more processor, a propensity model for a key performance indicator (KPI) of interest using the training data formed from combining user behavior features and/or user attributes with known outcomes for the KPI of interest, the propensity model being a machine learning model;
applying, for a plurality of users and by the one or more processors, the propensity model to predict an outcome for the KPI of interest for each user;
in response to the applying of the propensity model to predict the outcome for the KPI of interest for each user of the plurality of users, generating, by the one or more processor, a user-reason code matrix that includes, for each user, of the plurality of users, a contribution value for each reason code from a plurality of reason codes, the contribution value indicates an impact of each reason code on a predicted outcome from the propensity model, each reason code representing a variable used by the propensity model to predict the outcome for the KPI of interest;
using the user-reason code matrix as input, applying, by the one or more processors, a topic model to generate a topic matrix and a user-topic matrix, the topic matrix including, for each topic from a plurality of topics, a contribution level for each of at least a portion of the plurality of reason codes indicating a rank of importance of each reason code in defining each topic, the user-topic matrix including, for each user, an attribution level for each topic from the plurality of topics based on the contribution values for the reason codes for each user and the contribution levels of reason codes for each topic;
generating, by the one or more processors, a plurality of user segments by clustering user data of each user, of the plurality of users, to one or more clusters based on the attribution levels for the plurality of topics for each user in the user-topic matrix, the generating of the plurality of user segments being indicative of assigning each user, of the plurality of users, to at least one topic, of the plurality of topics;
updating, by the one or more processors, the propensity model with training data from the predicted outcome that does not match an actual outcome; and
updating, by the one or more processors, the topic model with additional training data from the generated topic matrix and user-topic matrix.

2. The computer-implemented method of claim 1 , wherein each contribution value comprises a raw contribution score output by the propensity model.

3. The computer-implemented method of claim 1 , wherein each contribution value comprises an integer weight determined by comparing raw contribution scores output by the propensity model.

4. The computer-implemented method of claim 1 , further comprising:
utilizing the propensity model to generate a predicted outcome for a current user;
comparing the predicted outcome for the current user with an actual outcome for the current user; and
updating the propensity model when the predicted outcome for the current user fails to match the actual outcome for the current user.

5. The computer-implemented method of claim 1 , the method further comprising:
generating a descriptive segmentation results table based on the topic matrix and the user-topic matrix by assigning labels to each user segment.

6. The computer-implemented method of claim 1 , the method further comprising:
targeting a particular user associated with a first customer segment based on a subset of reason codes associated with the first customer segment.

7. The computer-implemented method of claim 1 , wherein the plurality of users are selected from a larger set of users based on the plurality of users having predicted outcomes within a predefined threshold level for the KPI of interest determined using the propensity model.

8. One or more computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising:
converting raw event data to training data by performing feature engineering;
based the converting, training a propensity model for a key performance indicator (KPI) of interest using the training data;
subsequent to the training, using the propensity model to analyze user behavior and/or user attributes to determine, for each user from a plurality of users, a predicted outcome for the key performance indicator (KPI) of interest and a contribution value for each variable from a plurality of variables used by the propensity model, the contribution value for each variable indicating an extent to which that variable contributes to the predicted outcome for each user, the propensity model being a machine learning model;
applying a topic model to the contribution values for the plurality of variables for the plurality of users to determine a plurality of topics, each topic comprising a group of variables with a contribution level for each variable indicating a rank of importance of each variable in defining the topic;
using the topic model to determine an attribution level for each user to each topic based on the contribution values for the variables for each user and the contribution levels of variables for each topic;
generating a plurality of user segments by clustering user data of each user, of the plurality of users, into one or more clusters, the clustering being indicative of assigning each user, of the plurality of users, to a topic from the plurality of topics based on the attribution levels for each user output by the topic model;
updating the propensity model with training data from the predicted outcome that does not match an actual outcome; and
updating the topic model with additional training data from one or more matrices.

9. The one or more computer storage media of claim 8 , wherein the operations further comprise:
wherein the feature engineering on the raw event data is used to generate user behavior features;
combining the user behavior features and user attributes with known outcomes to form the training data.

10. The one or more computer storage media of claim 8 , wherein each contribution value comprises a raw contribution score output by the propensity model.

11. The one or more computer storage media of claim 8 , wherein each contribution value comprises an integer weight determined by comparing raw contribution scores output by the propensity model.

12. The one or more computer storage media of claim 8 , wherein one or more features are determined not to contribute to the predicted outcome for each user from the plurality of users based on the propensity model, and the one or more features are excluded from the plurality of features.

13. The one or more computer storage media of claim 8 , the operations further comprising:
in response to using the propensity model for the predicted outcome, generating a user-reason code matrix that includes, for each user from the plurality of users, a reason code for each variable and the contribution value corresponding with each variable.

14. The one or more computer storage media of claim 8 , the operations further comprising;
generating a topic matrix that includes, for each topic from the plurality of topics, a reason code for each variable from the group of variables for the topic with the contribution level for each corresponding variable.

15. The one or more computer storage media of claim 8 , the o

Assignees

• Adobe Inc

Inventors

• Wu Xiang
• Yan Zhenyu
• Kuo Yi-Hong
• Dai Wuyang
• Bartik Polina
• Pani Abhishek

Classifications

• G06N20/00

  Machine learning · CPC title

• G06F17/16

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06Q10/067

  Enterprise or organisation modelling · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06Q30/0204 Primary

  Market segmentation · CPC title