Tag prediction for images or video content items

Patent metadata

Field	Value
Publication number	US-2016188592-A1
Application number	US-201414582731-A
Country	US
Kind code	A1
Filing date	Dec 24, 2014
Priority date	Dec 24, 2014
Publication date	Jun 30, 2016
Grant date	—

Abstract

Systems, methods, and non-transitory computer-readable media can create, in a training phase, a first content item representation of a first content item based on a first content item transformation. The first content item can comprise one or more of images and video. A first user metadata representation of first user metadata may be created based on a first user metadata transformation. The first content item representation and the first user metadata representation can be combined to produce a first combined representation. The first combined representation and a first tag representation of a first tag can be embedded in an embedding space within a first threshold distance from one another.

First claim

What is claimed is:

1 . A computer implemented method comprising:
in a training phase: creating a first content item representation of a first content item based on a first content item transformation, the first content item comprising one or more of images and video; creating a first tag representation of a first tag based on a first tag transformation, the first tag associated with the first content item; and embedding the first content item representation and the first tag representation in an embedding space within a first threshold distance from one another.

2 . The computer implemented method of claim 1 , further comprising:
training the first content item transformation and the first tag transformation so that the first content item representation and the first tag representation in the embedding space are embedded within the first threshold distance from one another.

3 . The computer implemented method of claim 1 , further comprising:
in an evaluation stage: creating a second content item representation of a second content item based on a second content item transformation, the second content item comprising one or more of images and video; embedding the second content item representation in the embedding space; and identifying at least one tag associated with the second content item in the embedding space within a second threshold distance from the second content item representation.

4 . The computer implemented method of claim 1 , wherein the first content item transformation or the first tag transformation is implemented using a matrix.

5 . The computer implemented method of claim 1 , wherein the first content item transformation or the first tag transformation comprises a linear transformation.

6 . The computer implemented method of claim 1 , wherein the first content item transformation or the first tag transformation comprises a nonlinear transformation.

7 . The computer implemented method of claim 1 , wherein creating the first content item representation of the first content item based on the first content item transformation comprises:
creating a content item vector corresponding to the content item; multiplying the content item vector by a content transformation matrix.

8 . The computer implemented method of claim 1 , wherein creating the first tag representation of the first tag based on the first tag transformation comprises:
creating a first tag vector corresponding to the first tag; multiplying the first tag vector by a first tag transformation matrix.

9 . The computer implemented method of claim 1 , wherein the first tag comprises a hashtag associated with the first content item.

10 . The computer implemented method of claim 1 , wherein the first content item comprises an image or video being uploaded to a social networking system.

11 . A system comprising:
at least one processor; a memory storing instructions configured to instruct the at least one processor to perform: in a training phase: creating a first content item representation of a first content item based on a first content item transformation, the first content item comprising one or more of images and video; creating a first tag representation of a first tag based on a first tag transformation, the first tag associated with the first content item; and embedding the first content item representation and the first tag representation in an embedding space within a first threshold distance from one another.

12 . The system of claim 11 , wherein the instructions are configured to instruct the at least one processor to perform:
training the first content item transformation and the first tag transformation so that the first content item representation and the first tag representation in the embedding space are embedded within the first threshold distance from one another.

13 . The system of claim 11 , wherein the instructions are configured to instruct the at least one processor to perform:
in an evaluation stage: creating a second content item representation of a second content item based on a second content item transformation, the second content item comprising one or more of images and video; embedding the second content item representation in the embedding space; and identifying at least one tag associated with the second content item in the embedding space within a second threshold distance from the second content item representation.

14 . The system of claim 11 , wherein the first content item transformation or the first tag transformation is implemented using a matrix.

15 . The system of claim 11 , wherein the first content item transformation or the first tag transformation comprises a linear transformation.

16 . A computer storage medium storing computer-executable instructions that, when executed, cause a computer system to perform a computer-implemented method comprising:
in a training phase: creating a first content item representation of a first content item based on a first content item transformation, the first content item comprising one or more of images and video; creating a first tag representation of a first tag based on a first tag transformation, the first tag associated with the first content item; embedding the first content item representation and the first tag representation in an embedding space within a first threshold distance from one another.

17 . The computer storage medium of claim 16 , wherein the computer-implemented method further comprises:
training the first content item transformation and the first tag transformation so that the first content item representation and the first tag representation in the embedding space are embedded within the first threshold distance from one another.

18 . The computer storage medium of claim 16 , wherein the computer-implemented method further comprises:
in an evaluation stage: creating a second content item representation of a second content item based on a second content item transformation, the second content item comprising one or more of images and video; embedding the second content item representation in the embedding space; and identifying at least one tag associated with the second content item in the embedding space within a second threshold distance from the second content item representation.

19 . The computer storage medium of claim 16 , wherein the first content item transformation or the first tag transformation is implemented using a matrix.

20 . The computer storage medium of claim 16 , wherein the first content item transformation or the first tag transformation comprises a linear transformation.

Assignees

• Facebook Inc

Inventors

• Fergus Robert D
• Bourdev Lubomir
• Denton Emily Lynn
• Weston Jason E

Classifications

• G06Q10/40 Primary

  Business processes related to social networking or social networking services · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06F16/48 Primary

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title