Adjusting pacing of notifications based on interactions with previous notifications

Patent metadata

Field	Value
Publication number	US-10313461-B2
Application number	US-201615354948-A
Country	US
Kind code	B2
Filing date	Nov 17, 2016
Priority date	Nov 17, 2016
Publication date	Jun 4, 2019
Grant date	Jun 4, 2019

Abstract

An online system predicts the user's activity on the online system during a future time interval (e.g., the next day). The system collects activity data, such as actions that the user has taken on the system. The system predicts whether the user is likely to be active during the future time interval based on features extracted from the activity data. This system determines selection of notifications and delivery of notifications based on the predicted time when the user is likely to be active on the online system. The system further records the user's past interactions with notifications, such as whether the user viewed the notification, whether the user interacted with a content item associated with the notification, and so on. If system determines a rate of delivery of notifications to the user based on the frequency of past user interactions with notifications.

First claim

What is claimed is:

1. A method comprising:
identifying a plurality of candidate notifications for sending to a user of an online system, each candidate notification associated with a content item posted on the online system;
for each of the plurality of candidate notifications:
extracting features associated with the candidate notification, wherein the features comprise a content type for the associated content item,
providing the features as input to a model generated using machine learning, the model trained using past user interactions with notifications sent to the user by the online system, and
obtaining, as an output from the model, a score indicating a predicted click-through rate for the candidate notification;

determining a likelihood of the user actively interacting with the online system during a future time interval;
responsive to the determined likelihood of the user actively interacting with the online system during the future time interval being greater than a threshold value, selecting one or more candidate notifications having a predicted click-through rate for the user exceeding a threshold click-through rate, the threshold click-through rate determined based on interactions performed by the user with notifications previously provided to the user; and
sending the one or more candidate notifications to a client device associated with the user prior to the future time interval, wherein the sending of the candidate notification is initiated at the online system.

2. The method of claim 1 , wherein selecting the one or more candidate notifications to send to the user comprises determining a number of candidate notifications to be sent to the user as a value not exceeding a notification limit.

3. The method of claim 2 , wherein the notification limit is a predetermined constant value.

4. The method of claim 2 , wherein the notification limit is determined based on the user's past interactions with notifications.

5. The method of claim 1 , wherein each candidate notification is further associated with a user performing an action associated with the content item associated with the candidate notification, and wherein the features further comprise a feature derived based on an identifier for the user performing the action.

6. The method of claim 1 , wherein each candidate notification is further associated with an entity represented in the online system to which the associated content item is posted, and wherein the features further comprise a feature derived based on an identifier for the entity.

7. The method of claim 6 , wherein the entity is one of a group or a brand page.

8. The method of claim 1 , further comprising:
for each past notification of a plurality of past notifications, each past notification previously provided to the user of the online system and associated with a content item on the online system, determining a set of features corresponding to the past notification, wherein the features comprise a content type for the associated content item; and
training the model using the sets of features associated with the plurality of past notifications.

9. A non-transitory computer-readable medium including instructions for execution on a processor, the instructions, when executed by the processor, causing the processor to:
identify a plurality of candidate notifications for sending to a user of an online system, each candidate notification associated with a content item posted on the online system;
for each of the plurality of candidate notifications:
extract features associated with the candidate notification, wherein the features comprise a content type for the associated content item,
provide the features as input to a model generated using machine learning, the model trained using past user interactions with notifications sent to the user by the online system, and
obtain, as an output from the model, a score indicating a predicted click-through rate for the candidate notification;

determine a likelihood of the user actively interacting with the online system during a future time interval;
responsive to the determined likelihood of the user actively interacting with the online system during the future time interval being greater than a threshold value, selecting one or more candidate notifications having a predicted click-through rate for the user exceeding a threshold click-through rate, the threshold click-through rate determined based on interactions performed by the user with notifications previously provided to the user; and
sending the one or more candidate notifications to a client device associated with the user prior to the future time interval, wherein the sending of the candidate notification is initiated at the online system.

10. The non-transitory computer-readable medium of claim 9 , wherein selecting the one or more candidate notifications to send to the user comprises determining a number of candidate notifications to be sent to the user as a value not exceeding a notification limit.

11. The non-transitory computer-readable medium of claim 10 , wherein the notification limit is a predetermined constant value.

12. The non-transitory computer-readable medium of claim 10 , wherein the notification limit is determined based on the user's past interactions with notifications.

13. The non-transitory computer-readable medium of claim 9 , wherein each candidate notification is further associated with a user performing an action associated with the content item associated with the candidate notification, and wherein the features further comprise a feature derived based on an identifier for the user performing the action.

14. The non-transitory computer-readable medium of claim 9 , wherein each candidate notification is further associated with an entity represented in the online system to which the associated content item is posted, and wherein the features further comprise a feature derived based on an identifier for the entity.

15. The non-transitory computer-readable medium of claim 14 , wherein the entity is one of a group or a brand page.

16. The non-transitory computer-readable medium of claim 9 , further comprising:
for each past notification of a plurality of past notifications, each past notification previously provided to the user of the online system and associated with a content item on the online system, determining a set of features corresponding to the past notification, wherein the features comprise a content type for the associated content item; and
training the model using the sets of features associated with the plurality of past notifications.

17. A system comprising:
a processor configured to execute instructions;
a non a non-transitory computer-readable medium including instructions for execution by the processor, the instructions causing the processor to:
identify a plurality of candidate notifications for sending to a user of an online system, each candidate notification associated with a content item posted on the online system;
for each of the plurality of candidate notifications:
extract features associated with the candidate notification, wherein the features comprise a content type for the associated content item,
provide the features as input to a model generated using machine learning, the model trained using past user interactions with notifications sent to the user by the online system, and
obtain, as an output from the model, a score indicating a predicted click-through rate for the candidate notification;

determine a likelihood of the user actively interacting with the online system

Assignees

• Facebook Inc

Inventors

• Ju Li
• Folz Matthew Bryan
• Palaniappan Ramanathan
• Ma Jonathan
• Bara Pawel

Classifications

• G06Q10/40

  Business processes related to social networking or social networking services · CPC title

• H04L67/306

  User profiles · CPC title

• G06F2201/81

  Threshold · CPC title

• G06Q30/0201 Primary

  Market modelling; Market analysis; Collecting market data · CPC title

• G06F11/34

  Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation {; Recording or statistical evaluation of user activity, e.g. usability assessment} · CPC title