Deterministic message distribution

Patent metadata

Field	Value
Publication number	US-10650325-B2
Application number	US-201514814555-A
Country	US
Kind code	B2
Filing date	Jul 31, 2015
Priority date	Jul 31, 2015
Publication date	May 12, 2020
Grant date	May 12, 2020

Abstract

This disclosure relates to systems and methods that include configuring a machine learning system to train on a plurality of messages, the machine learning system to output an expected number of positive responses and an expected number of negative responses based on an input message, determining a threshold differential and a weight value using responses to the plurality of messages, and sending the input message in response to a differential between the expected number of positive responses and the weight multiplied by the expected number of negative responses being above the threshold differential.

First claim

What is claimed is:

1. A system comprising:
a non-transitory machine-readable medium having instructions stored thereon, which, when executed by a processor, cause the system to:
configure a machine learning system to train on a plurality of training messages, wherein each individual instance of each training message has previously been sent to a first plurality of members of the online system, and each individual instance of each training message indicates one of a plurality of channels by which the individual instance of the training message was sent, and each individual instance of each training message is associated with one of either a positive response, negative response, or no response, from each member to whom the individual instance of the training message was sent, the machine learning system, subsequent to being trained and upon receiving an input message, to output an expected number of positive responses and an expected number of negative responses to be received from a second plurality of members, for each channel of the plurality of channels, based on the input message;
for the input message targeted for distribution to the second plurality of members of the online system, using the trained machine learning system to generate output indicating a number of positive responses expected from the second plurality of members and a number of negative responses expected from the second plurality of members, for each channel of the plurality of channels;
calculate, for each channel of the plurality of channels for the input message, a differential value by subtracting from the number of positive responses expected from the second plurality of members, the number of negative responses expected from the second plurality of members, the number of negative responses multiplied by a weighting factor prior to said subtracting;
select a channel for the input message that results in a highest differential value of the differential values calculated for the input message; and
send the input message to the second plurality of members via the selected channel in response to the highest differential value exceeding a threshold differential value.

2. The system of claim 1 , wherein the channel is selected from the group consisting of email, mobile text messaging, an application notification, and a profile message.

3. The system of claim 1 , wherein the training messages previously sent to the first plurality of members of the online system indicate a particular time of day at which they were received, and the machine learning system is to output an expected number of positive responses and an expected number of negative responses from the second plurality of members, for each of several different times of day, the instructions further cause the system to select a time of day to send the input message that results in a highest differential value between the expected number of positive responses and the expected number of negative responses multiplied by the weighting factor, the system sending the input message at the time of day in response to the highest differential value exceeding the threshold differential value.

4. The system of claim 1 , wherein the expected number of positive responses is based on at least one of page views, clicked links, purchases, likes, and comments, and the expected number of negative responses are based on at least one of unsubscribes, complaints, dislikes, and spam reports.

5. The system of claim 1 , wherein the machine learning system is configured to train on a user's activity for a threshold period of time based on a response to one of a plurality of messages.

6. The system of claim 5 , wherein the user's activity is tracked using the user's network address.

7. The system of claim 1 , wherein determining the threshold differential value and the weighting factor is based on a multi-objective optimization problem using constraints comprising a maximum number of negative responses and a minimum number of positive responses.

8. The system of claim 7 , wherein determining the threshold differential value and the weighting factor is accomplished by applying a set of estimated threshold differential values and estimated weighting factors and selecting the threshold differential value and the weighting factor from the set that minimizes the number of messages to send while satisfying the constraints.

9. A method comprising:
configuring a machine learning system to train on a plurality of training messages, wherein each individual instance of each training message has previously been sent to a first plurality of members of the online system, each individual instance of each training message indicates one of a plurality of channels by which the instance of the training message was sent, and each individual instance of each training message is associated with one of either a positive response, negative response, or no response, from the member to whom the individual instance of the training message was sent, the machine learning system, subsequent to being trained and upon receiving an input message, to output an expected number of positive responses and an expected number of negative responses to be received from a second plurality of members, for each channel of the plurality of channels, based on the input message;
for the input message targeted for distribution to the second plurality of members of the online system, using the trained machine learning system, generating output indicating a number of positive responses expected from the second plurality of members and a number of negative responses expected from the second plurality of members, if the input message is sent to the second plurality of members, for each channel of the plurality of channels;
calculating, for each channel of the plurality of channels for the input message, a differential value by subtracting from the number of positive responses expected from the second plurality of members, the number of negative responses expected from the second plurality of members, the number of negative responses multiplied by a weighting factor prior to said subtracting;
selecting a channel for the input message that results in a highest differential value of the differential values calculated for the input message; and
sending the input message to the second plurality of members via the selected channel in response to the highest differential value exceeding a threshold differential value.

10. The method of claim 9 , wherein the channel is selected from the group consisting of email, mobile text messaging, an application notification, and a profile message.

11. The method of claim 9 , wherein the training messages previously sent to the first plurality of members of the online system indicate a particular time of day at which they were received, and the machine learning system is to output an expected number of positive responses and an expected number of negative responses, for each of several different times of day, the method further comprising selecting a time of day to send the input message that results in a highest differential value between the expected number of positive responses and the expected number of negative responses multiplied by the weighting factor, the system sending the input message at the selected time of day in response to the highest differential value exceeding threshold differential value.

12. The method of claim 9 , wherein the expected number of positive responses is based on at least one of page views, clicked links, purchases, likes, and comments, and the expected number of negative responses are based on at least one of unsubscribes, complaints, dislikes, and spam reports.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Gupta Rupesh
• Tseng Hsiao-Ping
• Vijay Ravi Kiran Holur
• Rosales Romer E

Classifications

• G06N5/04

  Inference or reasoning models · CPC title

• H04L67/306

  User profiles · CPC title

• H04L67/2828

  Electricity · mapped topic

• H04L67/22

  Electricity · mapped topic

• H04L51/24

  Electricity · mapped topic