Personalizing explainable recommendations with bandits

Patent metadata

Field	Value
Publication number	US-11301513-B2
Application number	US-201916502975-A
Country	US
Kind code	B2
Filing date	Jul 3, 2019
Priority date	Jul 6, 2018
Publication date	Apr 12, 2022
Grant date	Apr 12, 2022

Abstract

Methods, systems and computer program products are provided personalizing recommendations of items with associated explanations. The example embodiments described herein use contextual bandits to personalize explainable recommendations (“recsplanations”) as treatments (“Bart”). Bart learns and predicts satisfaction (e.g., click-through rate, consumption probability) for any combination of item, explanation, and context and, through logging and contextual bandit retraining, can learn from its mistakes in an online setting.

First claim

What is claimed is:

1. A method for personalizing recommendations of items with associated explanations, comprising the steps of:
collecting a plurality of user attributes;
collecting, for each item of a plurality of media items, at least one item attribute;
collecting implicit feedback data based on at least one activity associated with at least one of the plurality of media items;
training a reward model based on the plurality of user attributes, the at least one item attribute, the implicit feedback data, and a plurality of candidate explanations;
defining, based on the reward model, a policy containing a distribution of explanations, a distribution of the plurality of media items, and a probability that each of the plurality of media items and explanations will be chosen;
generating an ordering of selectable items with associated explanations using the policy;
recording selection data corresponding to an item selected from the selectable items and explanations; and
updating the reward model with the recorded selection data.

2. The method according to claim 1 , further comprising the steps of:
recording the selection data at a time of selection of the selected item, the selection data including: at least one of the plurality of user attributes, the at least one item attribute, an item identifier, the implicit feedback data, and the probability that the policy placed on the chosen item and explanation for the selected item; and
updating the reward model with the recorded selection data.

3. The method according to claim 1 , wherein the plurality of media items is (i) a plurality of playlists, (ii) a plurality of songs, (iii) a plurality of albums, (iv) a plurality of artists, or any combination of (i), (ii), (iii), and (iv).

4. The method according to claim 1 , wherein the policy is conditioned on the plurality of user attributes, the at least one item attribute and a predicted reward for each item and explanation.

5. The method according to claim 1 , wherein the activity is at least one of a playback action and a save action.

6. The method according to claim 2 , further comprising the step of:
recording a playback length of the selected item.

7. The method according to claim 2 , further comprising the step of:
setting, after the selection of the selected item, the probability of at least one non-optimal item and corresponding explanation to greater than zero (0).

8. A computer-readable medium having stored thereon sequences of instructions, the sequences of instructions including instructions which when executed by a computer system causes the computer system to:
collect a plurality of user attributes;
collect, for each item of a plurality of media items, at least one item attribute;
collect implicit feedback data based on at least one activity associated with at least one of the plurality of media items;
train a reward model based on the plurality of user attributes, the at least one item attribute, the implicit feedback data, and a plurality of candidate explanations;
define, based on the reward model, a policy containing a distribution of explanations, a distribution of the plurality of media items, and a probability that each of the plurality of media items and explanations will be chosen;
generate an ordering of selectable items with associated explanations using the policy;
record selection data corresponding to an item selected from the selectable items and explanations; and
update the reward model with the recorded selection data.

9. A system for personalizing recommendations of items with associated explanations, comprising:
a user attribute database adapted to collect a plurality of user attributes;
an item attribute database adapted to collect, for each item of a plurality of media items, at least one item attribute;
an implicit feedback database adapted to collect implicit feedback data based on at least one activity associated with at least one of the plurality of media items;
at least one processor adapted to:
train a reward model based on the plurality of user attributes, the at least one item attribute, the implicit feedback data, and a plurality of candidate explanations;
define, based on the reward model, a policy containing a distribution of explanations, a distribution of the plurality of media items, and a probability that each of the plurality of media items and explanations will be chosen;
generate an ordering of selectable items with associated explanations using the policy; and
record selection data corresponding to an item selected from the selectable items and explanations; and
update the reward model with the recorded selection data.

10. The system according to claim 9 , further comprising:
a user activity log adapted to record the selection data at a time of selection of the selected item, the selection data including: at least one of the plurality of user attributes, the at least one item attribute, an item identifier, the implicit feedback data, and the probability that the policy placed on the chosen item and explanation for the selected item; and
the at least one processor is further adapted to update the reward model with the recorded selection data.

11. The system according to claim 9 , wherein the plurality of media items is (i) a plurality of playlists, (ii) a plurality of songs, (iii) a plurality of albums, (iv) a plurality of artists, or any combination of (i), (ii), (iii), and (iv).

12. The system according to claim 9 , wherein the policy is conditioned on the plurality of user attributes, the at least one item attribute and a predicted reward for each item and explanation.

13. The system according to claim 9 , wherein the activity is at least one of a playback action and a save action.

14. The system according to claim 10 , further comprising:
a user activity log adapted to record a playback length of the selected item.

15. The system according to claim 10 , wherein the at least one processor is further adapted to set, after the selection of the selected item, the probability of at least one non-optimal item and corresponding explanation to greater than zero (0).

16. The method according to claim 1 , further comprising the step of:
outputting the ordering of the selectable items with associated explanations.

17. The computer-readable medium according to claim 8 , wherein the instructions which when executed by the computer system further cause the computer system to:
output the ordering of selectable items with associated explanations.

18. The system according to claim 9 , further comprising:
a display to output the ordering of selectable items with associated explanations.

Assignees

• Spotify Ab

Inventors

• Mcinerney James E
• Lacker Benjamin
• Hansen Samantha
• Gruson Aloïs
• Mehrotra Rishabh
• Bouchard Hugues

Classifications

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N5/045

  Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence · CPC title

• G06N20/00

  Machine learning · CPC title

• G06N3/08

  Learning methods · CPC title