Utilizing a recommendation system approach to determine electronic communication send times

Patent metadata

Field	Value
Publication number	US-11038976-B2
Application number	US-201916564768-A
Country	US
Kind code	B2
Filing date	Sep 9, 2019
Priority date	Sep 9, 2019
Publication date	Jun 15, 2021
Grant date	Jun 15, 2021

Abstract

The present disclosure relates to systems, methods, and non-transitory computer readable media for determining send times for distributing digital content to client devices utilizing a recommendation system approach. For example, the disclosed systems can utilize a recommendation system model such as a matrix factorization model, a factorization machine model, and/or a neural network to implement collaborative filtering to generate predicted response rates for particular candidate send times. Based on the predicted response rates indicating likelihoods of receiving responses for particular send times, the disclosed system can generate a distribution schedule to provide electronic communications at one or more of the send times.

First claim

What is claimed is:

1. A computer-implemented method of determining electronic message send times for individual recipient devices using a recommendation system approach, the computer-implemented method comprising:
identifying a user feature record associated with a user and a plurality of historical feature records for a plurality of users;
utilizing a factorization machine model to generate, based on the user feature record and the plurality of historical feature records, a predicted response rate schedule for the user by:
determining user feature vectors associated with the plurality of users, a first send time feature vector associated with a first candidate send time, and a second send time feature vector associated with a second candidate send time;
determining user-time feature interactions based on the user feature vectors, the first send time feature vector, and the second send time feature vector; and
determining, based on the user-time feature interactions, a first binary prediction of whether or not the user will respond at the first candidate send time and a second binary prediction of whether or not the user will respond at the second candidate send time;

generating, based on the predicted response rate schedule, a distribution schedule for providing electronic communications to the user; and
providing an electronic communication to the user based on the distribution schedule.

2. The computer-implemented method of claim 1 , further comprising determining a time granularity for bucketing candidate send times.

3. The computer-implemented method of claim 1 , wherein the user feature record comprises at least two of: a user identification associated with the user, an age of the user, a location associated with the user, and a domain associated with the user.

4. The computer-implemented method of claim 1 , wherein the predicted response rate schedule comprises one or more of a predicted click rate schedule, a predicted open rate schedule, or a predicted conversion rate schedule.

5. The computer-implemented method of claim 1 , wherein the predicted response rate schedule comprises a plurality of response rates associated with send times for distributing electronic communications to the user.

6. A non-transitory computer readable medium comprising instructions that, when executed by at least one processor, cause a computer device to:
identify a user feature record associated with a user and a plurality of historical feature records for a plurality of users;
utilize a matrix factorization model to generate, based on the user feature record and the plurality of historical feature records, a predicted response rate schedule for the user by:
generating a user-time identification matrix comprising user identifications for the plurality of users, a first candidate send time identification for a first candidate send time, and a second candidate send time identification for a second candidate send time;
decomposing the user-time identification matrix into a user identification matrix comprising the user identifications and a send time identification matrix comprising the first send time identification and the second send time identification; and
determining a first response rate for the first candidate send time and a second response rate for the second candidate send time based on respective combinations of a user identification matrix value and a send time identification matrix value;

generate, based on the predicted response rate schedule, a distribution schedule for providing electronic communications to the user; and
provide an electronic communication to the user based on the distribution schedule.

7. The non-transitory computer readable medium of claim 6 , further comprising instructions that, when executed by the at least one processor, cause the computing device to generate the predicted response rate schedule by further determining additional predicted response rates by send time that indicate probabilities that the user will respond to electronic communications distributed at respective send times.

8. The non-transitory computer readable medium of claim 6 , wherein the user-time identification matrix indicates cross-sections of the user identifications for the plurality of users and corresponding send time identifications for candidate send times.

9. The non-transitory computer readable medium of claim 6 , further comprising instructions that, when executed by the at least one processor, cause the computer device to rank the first candidate send time and the second candidate send time on a user-by-user basis for the user and for other users utilizing an area under a curve evaluation metric.

10. The non-transitory computer readable medium of claim 6 , further comprising instructions that, when executed by the at least one processor, cause the computer device to determine measures of likelihood that the user will respond at the first candidate send time and the second candidate send time utilizing an area under a curve evaluation metric.

11. The non-transitory computer readable medium of claim 6 , wherein the predicted response rate schedule comprises one or more of a predicted click rate schedule, a predicted open rate schedule, or a predicted conversion rate schedule.

12. The non-transitory computer readable medium of claim 6 , wherein the user feature record comprises at least two of: a user identification associated with the user, an age of the user, a location associated with the user, and a domain associated with the user.

13. The non-transitory computer readable medium of claim 6 , further comprising instructions that, when executed by the at least one processor, cause the computer device to determine a time granularity for bucketing candidate send times.

14. A system comprising:
one or more memory devices comprising a neural network; and
one or more server devices that cause the system to:
identify, for a user, a user identification, a user feature record, and a plurality of candidate send times for distributing electronic communications to the user;
utilize a neural network to generate, for the plurality of candidate send times, a predicted response rate schedule for the user corresponding to the user identification by:
determining a user embedding for the user, a first send time embedding for a first candidate send time, and a second time embedding for a second candidate send time;
generating a combination of the user embedding with one or more of the first send time embedding or the second send time embedding; and
applying, based on the combination, the neural network to generate a first response rate for the first candidate send time and a second response rate for the second candidate send time;

generate, based on the predicted response rate schedule, a distribution schedule for providing electronic communications to the user; and
provide an electronic communication to the user based on the distribution schedule.

15. The system of claim 14 , wherein the one or more server devices further cause the system to determine the response rates via collaborative filtering by determining relationships in a latent space between latent representations of the user feature record and historical feature records associated with the other users.

16. The system of claim 14 , wherein the one or more server devices further cause the system to identify a plurality of user feature records for a plurality of users, wherein a user feature record comprises a user identification and indications of historical responses to previously distributed electronic communications.

Assignees

• Adobe Inc

Inventors

• Liu Xinyue
• He Jun
• Yan Zhenyu
• Dai Wuyang
• Pani Abhishek

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• H04L67/55 Primary

  Push-based network services · CPC title

• H04L67/62

  Establishing a time schedule for servicing the requests · CPC title