Subject line tester

Patent metadata

Field	Value
Publication number	US-11475207-B2
Application number	US-202016901656-A
Country	US
Kind code	B2
Filing date	Jun 15, 2020
Priority date	Nov 21, 2019
Publication date	Oct 18, 2022
Grant date	Oct 18, 2022

Abstract

Methods, systems, and devices supporting data processing are described. In some systems, a data processing platform may support communication message analysis using machine learning. For example, a system may receive a set of communication messages (e.g., social media messages) and perform a machine learning process on the message contents and message interaction data to train a machine learned model. The system may further receive a subject line for a communication message for analysis, input the subject line into the machine learned model, and receive, as an output of the machine learned model, an engagement score based on the subject line. The engagement score may indicate an estimated probability that a user receiving the communication message opens the communication message (e.g., based on the subject line). A user—or the system—may modify the subject line based on the analysis to improve the engagement score.

First claim

What is claimed is:

1. A method for communication message analysis using machine learning, comprising:
receiving a plurality of social media messages comprising a plurality of respective message contents and corresponding to respective interaction data indicating how users interact with a social media message on a social media platform;
performing a machine learning process on the plurality of respective message contents and the respective interaction data corresponding to the plurality of social media messages to generate a machine learned model;
receiving a subject line for a communication message for analysis;
inputting the subject line into the machine learned model; and
receiving, as an output of the machine learned model, an engagement score based at least in part on the subject line, wherein the engagement score indicates an estimated probability that a user receiving the communication message on an email platform different from the social media platform opens the communication message.

2. The method of claim 1 , further comprising:
receiving, as an additional output of the machine learned model, one or more suggested changes to the subject line.

3. The method of claim 2 , further comprising:
receiving a user input indicating a suggested change of the one or more suggested changes to the subject line; and
updating the communication message to comprise an updated subject line based at least in part on the subject line and the suggested change.

4. The method of claim 2 , wherein receiving, as the additional output of the machine learned model, the one or more suggested changes to the subject line further comprises:
receiving one or more additional engagement scores corresponding to the one or more suggested changes to the subject line.

5. The method of claim 1 , further comprising:
receiving, as an additional output of the machine learned model, an indicated segmentation of the subject line into a plurality of portions and a respective indication of how each portion of the plurality of portions affects the engagement score for the subject line.

6. The method of claim 1 , further comprising:
sending, for display in a user interface of a user device, the engagement score.

7. The method of claim 1 , further comprising:
transmitting the communication message to one or more users;
receiving, from at least one user of the one or more users, feedback information indicating user engagement with the communication message; and
updating the machine learned model based at least in part on the feedback information.

8. The method of claim 7 , wherein the user engagement with the communication message comprises opening the communication message, replying to the communication message, forwarding the communication message, performing an action associated with the communication message, or a combination thereof.

9. The method of claim 7 , wherein the engagement score comprises a predicted user engagement score, the method further comprising:
determining an actual user engagement score based at least in part on the feedback information, wherein the machine learned model is updated based at least in part on a difference between the predicted user engagement score and the actual user engagement score.

10. The method of claim 1 , wherein performing the machine learning process further comprises:
determining an association between the plurality of respective message contents and the respective interaction data corresponding to the plurality of social media messages, wherein the machine learned model is based at least in part on the association.

11. The method of claim 1 , further comprising:
filtering a total set of social media messages to remove one or more outlier social media messages with message engagement data greater than a first threshold engagement amount or lower than a second threshold engagement amount, wherein the plurality of social media messages is received from the filtered total set of social media messages.

12. The method of claim 1 , wherein performing the machine learning process further comprises:
training the machine learned model using a first subset of the plurality of social media messages; and
validating the machine learned model using a second subset of the plurality of social media messages.

13. The method of claim 12 , wherein validating the machine learned model comprises:
inputting respective message contents for each message of the second subset of social media messages into the machine learned model to obtain respective predicted message interaction data for each message of the second subset of social media messages;
comparing the respective predicted message interaction data for each message of the second subset of social media messages to respective actual message interaction data corresponding to each message of the second subset of social media messages; and
further training the machine learned model based at least in part on the comparing.

14. The method of claim 1 , wherein receiving the plurality of social media messages further comprises:
identifying a criterion for an account based at least in part on a number of other accounts connected to the account, a number of social media messages associated with the account, content of social media messages associated with the account, or a combination thereof; and
selecting a plurality of accounts based at least in part on the criterion, wherein the plurality of social media messages is received from the selected plurality of accounts.

15. The method of claim 1 , wherein receiving the plurality of social media messages further comprises:
identifying a first social media message of the plurality of social media messages with a highest message interaction rate and a second social media message of the plurality of social media messages with a lowest message interaction rate, wherein the first social media message and the second social media message correspond to a range for the engagement score.

16. The method of claim 1 , wherein the plurality of social media messages is received based at least in part on a set of users generating the plurality of social media messages, a common subset of message contents of the plurality of social media messages, the respective interaction data corresponding to the plurality of social media messages, or a combination thereof.

17. The method of claim 1 , wherein the respective interaction data comprises a number of shares, a number of responses, a number of views, or a combination thereof.

18. The method of claim 1 , wherein:
the plurality of respective message contents comprises text, a link, an emoticon, an image, or a combination thereof;
the communication message comprises an email message; and
the subject line comprises text, a link, or a combination thereof.

19. An apparatus for communication message analysis using machine learning, comprising:
a processor;
memory coupled with the processor; and
instructions stored in the memory and executable by the processor to cause the apparatus to:
receive a plurality of social media messages comprising a plurality of respective message contents and corresponding to respective interaction data indicating how users interact with a social media message on a social media platform;
perform a machine learning process on the plurality of respective message contents and the respective interaction data corresponding to the plurality of social media messages to generate a machine learned model;
receive a subject line for a communication message for ana

Assignees

• Salesforce Com Inc
• Salesforce Inc

Inventors

• Xie Kexin
• Cagrici Gokhan
• Wilson Daniel Keith
• Mallick Shrestha Basu
• Belkowitz Jonathan Daniel Showers
• Lestina Jason
• Brewer James
• Gasperut Daniel Louis
• Zickgraf Jeffery Allen
• Lyman Greg
• Brewer Michael Ronald
• Black Evan
• Rauschuber Austin
• Schultz Victoria
• Trepina Matthew David
• Stadlinger Peter

Classifications

• H04L51/52

  for supporting social networking services · CPC title

• G06N20/00

  Machine learning · CPC title

• H04L51/063

  Content adaptation, e.g. replacement of unsuitable content · CPC title

• G06F40/166 Primary

  Editing, e.g. inserting or deleting · CPC title

• G06F40/216

  using statistical methods · CPC title