Differentiable user-item co-clustering

Patent metadata

Field	Value
Publication number	US-11636394-B2
Application number	US-202016912453-A
Country	US
Kind code	B2
Filing date	Jun 25, 2020
Priority date	Jun 25, 2020
Publication date	Apr 25, 2023
Grant date	Apr 25, 2023

Abstract

The present concepts relate to a differentiable user-item co-clustering (“DUICC”) model for recommendation and co-clustering. Users' interaction with items (e.g., content) may be centered around information co-clusters—groups of items and users that exhibit common consumption behavior. The DUICC model may learn fine-grained co-cluster structures of items and users based on their interaction data. The DUICC model can then leverage the learned latent co-cluster structures to calculate preference stores of the items for a user. The top scoring items may be presented to the user as recommendations.

First claim

The invention claimed is:

1. A system, comprising:
a processor;
storage storing instructions which, when executed by the processor, cause the processor to:
receive interactions between a plurality of users and a plurality of items;
based on the interactions, train a machine learning model to detect a plurality of co-clusters of the plurality of users and the plurality of items by:
generating a plurality of item embeddings representing the plurality of items;
generating a plurality of user embeddings representing the plurality of users based on weighted averages of the plurality of item embeddings by assigning weights to the plurality of items based on predictive strengths of the plurality of items, the weights varying across the plurality of co-clusters; and
calculating a plurality of item-co-cluster affinity scores of the plurality of items to the plurality of co-clusters and a plurality of user-co-cluster affinity scores of the plurality of users to the plurality of co-clusters based on the plurality of item embeddings, the plurality of user embeddings, and a plurality of co-cluster embeddings representing the plurality of co-clusters; and

output the plurality of item-co-cluster affinity scores and the plurality of user-co-cluster affinity scores.

2. The system of claim 1 , wherein the machine learning model uses an attention head for each of the plurality of co-clusters to generate the plurality of user embeddings.

3. The system of claim 1 , wherein the plurality of user embeddings are generated using a linear projection.

4. The system of claim 1 , wherein the machine learning model detects the plurality of co-clusters by:
calculating the plurality of item-co-cluster affinity scores based on the plurality of item embeddings and the plurality of co-cluster embeddings; and
calculating the plurality of user-co-cluster affinity scores based on the plurality of user embeddings and the plurality of co-cluster embeddings.

5. The system of claim 4 , wherein:
calculating an item-co-cluster affinity score of the plurality of item-co-cluster affinity scores for an item of the plurality of items and a co-cluster of the plurality of co-clusters comprises calculating a dot product of an item embedding of the plurality of item embeddings representing the item and a co-cluster embedding of the plurality of co-cluster embeddings representing the co-cluster; and
calculating a user-co-cluster affinity score of the plurality of user-co-cluster affinity scores for a user of the plurality of users and the co-cluster comprises calculating a dot product of a user embedding of the plurality of user embeddings representing the user and the co-cluster embedding.

6. The system of claim 1 , wherein the machine learning model is trained using a softmax classification loss function.

7. The system of claim 6 , wherein the machine learning model is trained by:
holding out an item in a set of items that a user of the plurality of users has interacted with; and
using the set of items that have not been held out to train the machine learning model using the softmax classification loss function to correctly predict the item that has been held out.

8. The system of claim 1 , wherein instructions further cause the processor to:
calculate a cached scalar value based on the plurality of item embeddings,
wherein the plurality of user embeddings are generated based on the cached scalar value.

9. The system of claim 8 , wherein the instructions further cause the processor to:
receive a new interaction between a user of the plurality of users and an item of the plurality of items;
recalculate the cached scalar value based on an item embedding of the plurality of item embeddings associated with the item; and
update a user embedding of the plurality of user embeddings associated the user based on the item embedding and the recalculated cached scalar value,
wherein a computational time for updating the user embedding is in an order of constant time with respect to a number of the interactions.

10. The system of claim 1 , wherein the instructions further cause the processor to implement an itemset aggregator module that generates the plurality of user embeddings, parameters of the itemset aggregator module being shared across the plurality of users, the itemset aggregator module consuming constant memory with respect to a number of the plurality of users.

11. A hardware computer readable storage medium storing instructions which, when executed by a processor, cause the processor to:
train a machine learning model that detects co-clusters of a plurality of users and a plurality of items by:
receiving interactions between the plurality of users and the plurality of items;
generate item embeddings based on the interactions;
generate a plurality of user embeddings based on weighted sums of the items embeddings by assigning weights to the plurality of items based on predictive strengths of the plurality of items, the weights varying across the co-clusters;
learn co-cluster embeddings based on the item embeddings and the plurality of user embeddings; and
determine item-co-cluster affinity scores and user-co-cluster affinity scores based on the plurality of user embeddings, the item embeddings, and the co-cluster embeddings;

receive a user identification associated with a user of the plurality of users;
calculate preference scores associated with the plurality of items for the user based on the user-co-cluster affinity scores and the item-co-cluster affinity scores determined by the machine learning model; and
output a set of the plurality of items for the user based on the preference scores.

12. The hardware computer readable storage medium of claim 11 , wherein the instructions further cause the processor to:
receive a new interaction between the user and an item of the plurality of items;
recalculate a cached scalar value based on an item embedding of the plurality of item embeddings associated with the item; and
update a user embedding of the plurality of user embeddings representing the user based on the item embedding and the recalculated cached scalar value using the machine learning model.

13. The hardware computer readable storage medium of claim 12 , wherein a computational time for updating the user embedding is in an order of constant time with respect to a number of the interactions.

14. The hardware computer readable storage medium of claim 11 , wherein the instructions further cause the processor to:
receive a new interaction between the user and an item of the plurality of items;
recalculate a cached scalar value based on an item embedding of the plurality of item embeddings associated with the item; and
update a user embedding of the plurality of user embeddings associated the user based on the item embedding and the recalculated cached scalar value,
wherein a computational time for updating the user embedding is in an order of constant time with respect to a number of the interactions.

15. The hardware computer readable storage medium of claim 11 , wherein the instructions further cause the processor to implement an itemset aggregator module that generates the plurality of user embeddings, parameters of the itemset aggregator module being shared across the plurality of users, the itemset aggregator module consuming constant memory with respect to a number of the plurality of users.

16. A method, comprising:
detecting co-clusters of a plurality of users and a plurality of items from interactions between the plurality of users and the plurality of items using a machine learning mo

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Yang Longqi
• Schnabel Tobias Benjamin
• Bennett Paul Nathan
• Dumais Susan Theresa

Classifications

• G06N3/048

  Activation functions · CPC title

• G06Q30/0631

  Recommending goods or services · CPC title

• G06Q30/0201

  Market modelling; Market analysis; Collecting market data · CPC title

• G06Q30/0251

  Targeted advertisements · CPC title

• G06N20/00 Primary

  Machine learning · CPC title