Content type embeddings

Patent metadata

Field	Value
Publication number	US-12099566-B2
Application number	US-201916675671-A
Country	US
Kind code	B2
Filing date	Nov 6, 2019
Priority date	Sep 23, 2019
Publication date	Sep 24, 2024
Grant date	Sep 24, 2024

Abstract

Techniques for learning and using content type embeddings. The content type embeddings have the useful property that a distance in an embedding space between two content type embeddings corresponds to a semantic similarity between the two content types represented by the two content type embeddings. The closer the distance in the space, the more the two content types are semantically similar. The farther the distance in the space, the less the two content types are semantically similar. The learned content type embeddings can be used in a content suggestion system as machine learning features to improve content suggestions to end-users.

First claim

The invention claimed is:

1. A computer-implemented method comprising:
generating a corpus of entity sequences comprising a plurality of content items in a content management system that are associated with a plurality of content types and that satisfy a recency heuristic indicating a recency of access within a period of time;
filtering the corpus of entity sequences to generate filtered entity sequences comprising a subset of the plurality of content items by removing non-content item entities from the corpus of entity sequences;
determining a plurality of pairwise co-occurrence instance counts for the plurality of content types reflecting pairwise co-occurrence instances among the plurality of content types in the filtered entity sequences;
generating a plurality of content type embeddings for the filtered entity sequences based on the plurality of pairwise co-occurrence instance counts;
counting, as one of the plurality of pairwise co-occurrence instance counts, a co-occurrence instance count for a first instance of a pair of different content types co-occurring in a same content item sharing operation facilitated by the content management system;
counting, as one of the plurality of pairwise co-occurrence instance counts, a co-occurrence instance count for a second instance of a pair of different content types co-occurring in a folder of a centrally-hosted network filesystem of the content management system based on the pair of different content types being contained in the folder of the centrally-hosted network filesystem for an amount of time;
weighing, in the plurality of pairwise co-occurrence instance counts, the co-occurrence instance count for first instance and the co-occurrence instance count for second instance numerically differently based on a type of co-occurrence instance count for the first instance and a type of co-occurrence instance count for the second instance;
generating magnitudes of distances in a multi-dimensional embedding space between the plurality of content type embeddings corresponding to a semantic similarity between the filtered entity sequences; and
providing, for display on a client device and based on the magnitudes of distances between the plurality of content type embeddings, a content item suggestion that references at least one particular content item.

2. The computer-implemented method of claim 1 , further comprising:
counting, as one of the plurality of pairwise co-occurrence instance counts, an instance of a pair of different content types co-occurring in a folder of a centrally-hosted network filesystem of the content management system based on the pair of different content types being contained in the folder of the centrally-hosted network filesystem for a threshold amount of time, wherein the centrally-hosted network filesystem is hierarchical with multiple folders located hierarchically with a parent folder and a child folder and the pair of different content types are contained in the folder of the centrally-hosted network filesystem for a threshold amount of time.

3. The computer-implemented method of claim 1 , further comprising counting, as one of the plurality of pairwise co-occurrence instance counts, an instance of a pair of different content types that are not functionally integrated co-occurring in a same content item upload session or synchronization session associated with the content management system based on being uploaded or synchronized within a threshold amount of time.

4. The computer-implemented method of claim 1 , further comprising counting, as one of the plurality of pairwise co-occurrence instance counts, an instance of a pair of different content types co-occurring in a same filtered translated entity sequence generated from a content management system graph, or a sub-graph thereof.

5. The computer-implemented method of claim 1 , further comprising learning the plurality of content type embeddings based on a generative model according to a generative process.

6. The computer-implemented method of claim 1 , further comprising using a particular content type embedding of the plurality of content type embeddings as an input machine learning feature to a machine learning content suggestion model to make a prediction about a particular content item, the particular content item being a particular content type, the particular content type represented by the particular content type embedding.

7. The computer-implemented method of claim 1 , wherein determining the plurality of pairwise co-occurrence instance counts for the plurality of content types is based on a random sampling of co-occurrence data collected during a period of time.

8. One or more non-transitory computer-readable media storing one or more computer programs having instructions which, when executed by a computing system having one or more processors, cause the computing system to:
generate filtered entity sequences comprising a subset of a plurality of content items by filtering a corpus of entity sequences to satisfy a recency heuristic indicating a recency of access within a period of time and filtering by removing non-content item entities from the corpus of entity sequences, wherein the corpus of entity sequences comprises a plurality of content items in a content management system that are associated with a plurality of content types;
generate a plurality of content type embeddings for the filtered entity sequences based on pairwise co-occurrence instances among the plurality of content types in the filtered entity sequences;
count, as one of the pairwise co-occurrence instances, a co-occurrence instance for a first instance of a pair of different content types co-occurring in a same content item sharing operation facilitated by the content management system;
count, as one of the pairwise co-occurrence instances, a co-occurrence instance count for a second instance of a pair of different content types co-occurring in a folder of a centrally-hosted network filesystem of the content management system based on the pair of different content types being contained in the folder of the centrally-hosted network filesystem for an amount of time;
weigh, in the pairwise co-occurrence instances, the co-occurrence instance for first instance and the co-occurrence instance for second instance numerically differently based on a type of co-occurrence instance for the first instance and a type of co-occurrence instance for the second instance;
train a machine learning content suggestion model based on a set of training examples, the set of training examples having the plurality of content type embeddings as machine learning features of the set of training examples, wherein a magnitude of a distance in a multi-dimensional embedding space between a pair of content type embeddings, of the plurality of content type embeddings, corresponds to a semantic similarity between a pair of content types, of the plurality of content types, represented by the pair of content type embeddings; and
use a particular content type embedding of the plurality of content type embeddings as an input machine learning feature to a machine learning content suggestion model to make a prediction about a particular content item, the particular content item being a particular content type, the particular content type represented by the particular content type embedding.

9. The one or more non-transitory computer-readable media of claim 8 , further comprising instructions, that when executed by the computing system, cause the computing system to:
store a plurality of pairwise co-occurrence instance counts for a plurality of content types reflecting pairwise co-occurrence instances in a content management system among the plurality of content types

Assignees

• Dropbox Inc

Inventors

• Baek Jongmin
• Ding Jiarui
• Kumar Neeraj

Classifications

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06F18/24137

  Distances to cluster centroïds · CPC title

• G06N3/08

  Learning methods · CPC title