Generating digital event sequences utilizing a dynamic user preference interface to modify recommendation model reward functions

Patent metadata

Field	Value
Publication number	US-11085777-B2
Application number	US-201816047908-A
Country	US
Kind code	B2
Filing date	Jul 27, 2018
Priority date	Jul 27, 2018
Publication date	Aug 10, 2021
Grant date	Aug 10, 2021

Abstract

The present disclosure relates to generating and modifying recommended event sequences utilizing a dynamic user preference interface. For example, in one or more embodiments, the system generates a recommended event sequence using a recommendation model trained based on a plurality of historical event sequences. The system then provides, for display via a client device, the recommendation, a plurality of interactive elements for entry of user preferences, and a visual representation of historical event sequences. Upon detecting input of user preferences, the system can modify a reward function of the recommendation model and provide a modified recommended event sequence together with the plurality of interactive elements. In one or more embodiments, as a user enters user preferences, the system additionally modifies the visual representation to display subsets of the plurality of historical event sequences corresponding to the preferences.

First claim

What is claimed is:

1. A computer-implemented method for generating and providing digital sequence recommendations based on interactive user interface preference elements, comprising:
generating a recommended event sequence by utilizing a recommendation model trained to iteratively utilize a Markov Decision Process model in conjunction with Thompson sampling to generate a plurality of events in a sequentially ordered combination based on a plurality of historical event sequences, wherein the recommendation model comprises a reward function and the plurality of historical event sequences comprises events in sequentially ordered combinations associated with a plurality of users;
providing, for display via a client device, a user interface displaying the recommended event sequence and a plurality of interactive elements for entry of user preferences; and
iteratively generating, for display within the user interface, modified recommended event sequences comprising events in sequentially ordered combinations by modifying the reward function of the recommendation model based on detecting selection of one or more user preferences via the plurality of interactive elements within the user interface.

2. The method of claim 1 , further comprising iteratively providing the modified recommended event sequences for display via the client device with the plurality of interactive elements.

3. The method of claim 1 , further comprising providing, for display via the client device, a visual representation of historical event sequences with the recommended event sequence within the user interface.

4. The method of claim 3 ,
wherein the visual representation of historical event sequences comprises a subset of the plurality of historical event sequences, and
further comprising iteratively modifying the visual representation of historical event sequences to display subsets of the plurality of historical event sequences corresponding to the one or more user preferences.

5. A non-transitory computer readable storage medium comprising instructions that, when executed by at least one processor, cause a computing device to:
generate a recommended event sequence by utilizing a recommendation model trained to iteratively utilize a Markov Decision Process model in conjunction with Thompson sampling to generate a plurality of events in a sequentially ordered combination based on a plurality of historical event sequences, wherein the recommendation model comprises a reward function and the plurality of historical event sequences comprises events in sequentially ordered combinations associated with a plurality of users;
provide, for display via a client device, a user interface comprising the recommended event sequence, a visual representation of historical event sequences, and a plurality of interactive elements for entry of user preferences;
detect input of user preferences via the plurality of interactive elements;
modify the reward function of the recommendation model based on the input of the user preferences;
generate a modified recommended event sequence based on the recommendation model and the modified reward function by modifying at least one event from the plurality of events of the recommended event sequence; and
provide the modified recommended event sequence for display via the client device with the plurality of interactive elements for additional entry of user preferences.

6. The non-transitory computer readable storage medium of claim 5 , further comprising instructions that, when executed by the at least one processor, cause the computing device to modify the visual representation of historical event sequences to display a subset of the plurality of historical event sequences corresponding to the user preferences.

7. The non-transitory computer readable storage medium of claim 5 , wherein the visual representation of historical event sequences comprises:
a plurality of rows, wherein each row represents a historical event sequence of the plurality of historical event sequences taken by a user; and
a plurality of columns, wherein each column represents an event within the historical event sequence.

8. The non-transitory computer readable storage medium of claim 5 ,
wherein the plurality of interactive elements further comprises a set of interactive elements for entry of user constraints, and
further comprising instructions that, when executed by the at least one processor, cause the computing device to generate the modified recommended event sequence further based on input of one or more user constraints via the set of interactive elements.

9. The non-transitory computer readable storage medium of claim 5 , wherein:
the recommended event sequence comprises a sequence of points of interest to visit, and
a user preference of the user preferences comprises one of a preferred point of interest category or a preferred point of interest.

10. The non-transitory computer readable storage medium of claim 5 , wherein:
the recommended event sequence comprises a sequence of digital content transmissions across client devices, and
a user preference of the user preferences comprises one of a distribution channel, a digital content category, or a digital content item for transmission.

11. The non-transitory computer readable storage medium of claim 5 , wherein the instructions, when executed by the at least one processor, cause the computing device to provide the recommended event sequence for display with the modified recommended event sequence via the client device.

12. The non-transitory computer readable storage medium of claim 5 , wherein the instructions, when executed by the at least one processor, cause the computing device to provide the modified recommended event sequence for display via the client device by:
providing a first visual indicator representing historical user responses corresponding to an event of the modified recommended event sequence; and
providing a second visual indicator representing a type of the event.

13. A system comprising:
at least one processor; and
at least one non-transitory computer readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:
generate a recommended event sequence by utilizing a recommendation model trained to iteratively utilize a Markov Decision Process model in conjunction with Thompson sampling to generate a plurality of events in a sequentially ordered combination based on a plurality of historical event sequences, wherein the recommendation model comprises a reward function and the plurality of historical event sequences comprises events in sequentially ordered combinations associated with a plurality of users;
provide, for display via a client device, a user interface displaying the recommended event sequence, a visual representation of historical event sequences, and a plurality of interactive elements for entry of user preferences;
generate preference weights based on detected input of one or more user preferences via the plurality of interactive elements;
modify the reward function of the recommendation model by applying the preference weights to the reward function;
generate a modified recommended event sequence utilizing the recommendation model and the modified reward function by modifying at least one event from the plurality of events of the recommended event sequence; and
provide the modified recommended event sequence for display via the client device with the plurality of interactive elements for additional entry of user preferences.

14. They system of claim 13 , further comprising instructions that, when e

Assignees

• Adobe Inc

Inventors

• Du Fan
• Lee Sana Malik
• Theocharous Georgios
• Koh Eunyee

Classifications

• G01C21/343 Primary

  Calculating itineraries (travelling salesman problem G06Q10/04; optimisation of routes G06Q10/047) · CPC title

• H04W4/021

  Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences · CPC title

• G01C21/3476

  using point of interest [POI] information, e.g. a route passing visible POIs · CPC title

• G01C21/3484

  Personalized, e.g. from learned user behaviour or user-defined profiles · CPC title

• H04W4/024

  Guidance services · CPC title