Utilizing a natural language model to determine a predicted activity event based on a series of sequential tokens

Patent metadata

Field	Value
Publication number	US-11567812-B2
Application number	US-202017065266-A
Country	US
Kind code	B2
Filing date	Oct 7, 2020
Priority date	Oct 7, 2020
Publication date	Jan 31, 2023
Grant date	Jan 31, 2023

Abstract

The present disclosure relates to systems, methods, and non-transitory computer-readable media that can leverage a natural language model to determine a most probable candidate sequence of tokens and thereby generate a predicted user activity. In particular, the disclosed systems can tokenize activity event vectors to generate a series of sequential tokens that correspond to recent user activity of one or more user accounts. In addition, the disclosed systems can, for each candidate (e.g., hypothetical) user activity, augment the series of sequential tokens to include a corresponding token. Based on respective probability scores for each of the augmented series of sequential tokens, the disclosed systems can identify as the predicted user activity, a candidate user activity corresponding to one of the augmented series of sequential tokens associated with a highest probability score. Based on the predicted user activity, the disclosed systems can surface one or more suggestions to a client device.

First claim

What is claimed is:

1. A system comprising:
at least one processor; and
at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:
identify, from an events database, a sequence of activity events, each activity event within the sequence of activity events comprising raw event data associated with a user account of a content management system;
generate a sequence of activity event vectors by generating an activity event vector for each activity event based on the raw event data for each respective activity event;
determine, utilizing a natural language processing (NLP) model, a predicted activity event based on the sequence of activity event vectors; and
perform an action based on the predicted activity event.

2. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to perform the action by providing, for display on a client device associated with the user account, one or more suggestions based on the predicted activity event.

3. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to generate the sequence of activity event vectors by:
generating feature embeddings for features from the raw event data of each activity event within the sequence of activity events;
concatenating a first set of feature embeddings corresponding to a first activity event to generate a first activity event vector; and
concatenating a second set of feature embeddings corresponding to a second activity event subsequent to the first activity event to generate a second activity event vector.

4. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:
generate, utilizing a hashing function, respective tokens for each activity event vector in the sequence of activity event vectors; and
generate, based on the respective tokens, a series of sequential tokens for the sequence of activity event vectors.

5. The system of claim 4 , further comprising instructions that, when executed by the at least one processor, cause the system to generate, utilizing the NLP model, the predicted activity event by:
determining, for a set of candidate sequences combining the series of sequential tokens with respective tokens of candidate activity events, a set of probability scores; and
selecting, as the predicted activity event, a candidate activity event corresponding to a candidate sequence associated with a highest probability score.

6. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to group the sequence of activity event vectors for generating a series of sequential tokens by grouping, into the sequence of activity event vectors, activity event vectors based on timestamps of corresponding feature embeddings.

7. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:
generate, from the events database, a corpus of series of sequential tokens associated with multiple user accounts of a content management system; and
apply the corpus of series of sequential tokens to the NLP model to identify one or more learned parameters.

8. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:
determine a set of predicted activity events based on the sequence of activity event vectors; and
perform multiple actions based on the set of predicted activity events, each action corresponding to a respective predicted activity event of the set of predicted activity events.

9. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to perform the action based on the predicted activity event by predicting a user segment for the user account.

10. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to identify, from the events database, the sequence of activity events by:
analyzing, from the events database, raw event data to identify timestamps for activity events for a threshold time period; and
determining, based on the identified timestamps, that a subset of the activity events occurring during the threshold time period corresponds to the sequence of activity events.

11. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computing device to:
identify, from an events database, a sequence of activity events, each activity event within the sequence of activity events comprising raw event data associated with a user account of a content management system;
generate a sequence of activity event vectors by generating an activity event vector for each activity event based on the raw event data for each respective activity event;
provide the sequence of activity event vectors to a natural language processing (NLP) model;
receive, from the NLP model, one or more predicted activity events; and
provide, to a client device associated with the user account, one or more suggestions based on the one or more predicted activity events.

12. The non-transitory computer-readable medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computing device to provide, for display on the client device associated with the user account, the one or more suggestions comprising:
a suggested team workspace accessible by the user account and one or more other user accounts of the content management system;
a suggested digital file to digitally share with the one or more other user accounts; or
a suggested access privilege or a suggested software tool based on a user-account classification for the user account.

13. The non-transitory computer-readable medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computing device to generate a sequence of activity event vectors by generating feature embeddings based on features from the raw event data of each activity event within the sequence of activity events.

14. The non-transitory computer-readable medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computing device to provide, for display on the client device associated with the user account, the one or more suggestions comprising an action suggestion that includes one or more of:
downloading, accessing, or editing a digital content item;
creating a calendar invite;
generating a digital reminder; or
generating a task list.

15. The non-transitory computer-readable medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computing device to provide the sequence of activity event vectors to the NLP model by generating a token for each activity event vector in the sequence of activity event vectors.

16. A computer-implemented method comprising:
identifying, from an events database, a sequence of activity events, each activity event within the sequence of activity events comprising raw event data associated with a group of user accounts of a content management system;
generating a sequence of activity event vectors by generating an activity event vector for each activity event based on the raw event data for eac

Assignees

• Dropbox Inc

Inventors

• Kulkarni Ranjitha Gurunath
• Xiang Xingyu
• Baek Jongmin
• Wei Ermo

Classifications

• G06F40/284 Primary

  Lexical analysis, e.g. tokenisation or collocates · CPC title

• G06F9/542 Primary

  Event management; Broadcasting; Multicasting; Notifications · CPC title

• G06N5/02

  Knowledge representation; Symbolic representation · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/09

  Supervised learning · CPC title