Scoring concept terms using a deep network

Patent metadata

Field	Value
Publication number	US-9514405-B2
Application number	US-201514860462-A
Country	US
Kind code	B2
Filing date	Sep 21, 2015
Priority date	Mar 13, 2013
Publication date	Dec 6, 2016
Grant date	Dec 6, 2016

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for scoring concept terms using a deep network. One of the methods includes receiving an input comprising a plurality of features of a resource, wherein each feature is a value of a respective attribute of the resource; processing each of the features using a respective embedding function to generate one or more numeric values; processing the numeric values to generate an alternative representation of the features of the resource, wherein processing the floating point values comprises applying one or more non-linear transformations to the floating point values; and processing the alternative representation of the input to generate a respective relevance score for each concept term in a pre-determined set of concept terms, wherein each of the respective relevance scores measures a predicted relevance of the corresponding concept term to the resource.

First claim

What is claimed is:

1. A method performed by one or more computers, the method comprising:
receiving an input comprising a plurality of features of a resource, wherein each of the features is of a different feature type;
generating an alternative representation of the features of the resource, comprising:
generating a respective numeric representation of each the features by processing each of the features using a respective embedding function, wherein each of the embedding functions is specific to features of a respective feature type, and
processing the respective numeric representations through one or more neural network layers to generate the alternative representation; and

providing the alternative representation of the features of the resource as input to a neural network classifier for classification of a relevance of a plurality of concept terms to the resource.

2. The method of claim 1 , wherein the neural network classifier is configured to:
process the alternative representation of the features of the resource to generate a respective relevance score for each of the plurality of concept terms, wherein each of the respective relevance scores measures a predicted relevance of the corresponding concept term to the resource.

3. The method of claim 1 , wherein the numeric representations are vectors of floating point values.

4. The method of claim 1 , wherein the numeric representations are vectors of quantized integer values, and wherein an encoding of the quantized integer values represents floating point values.

5. The method of claim 1 , further comprising:
obtaining the plurality of features of the resource in response to receiving an indication that an online advertisement auction is to be conducted to select one or more advertisements for inclusion in a particular presentation of the resource.

6. The method of claim 5 , further comprising:
selecting one or more of the concept terms as advertising keywords to be used in selecting candidate advertisements for participation in the online advertisement auction based on the classification of the relevance of the concept terms to the resource.

7. The method of claim 1 , wherein a particular feature of the plurality of features is not discrete, and wherein the method further comprises:
hashing the particular feature to generate a hashed feature;
partitioning the hashed feature into a particular partition of a predetermined set of partitions; and
processing a value corresponding to the particular partition using the embedding function for the feature.

8. A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:
receiving an input comprising a plurality of features of a resource, wherein each of the features is of a different feature type;
generating an alternative representation of the features of the resource, comprising:
generating a respective numeric representation of each the features by processing each of the features using a respective embedding function, wherein each of the embedding functions is specific to features of a respective feature type, and
processing the respective numeric representations through one or more neural network layers to generate the alternative representation; and

providing the alternative representation of the features of the resource as input to a neural network classifier for classification of a relevance of a plurality of concept terms to the resource.

9. The system of claim 8 , wherein the neural network classifier is configured to:
process the alternative representation of the features of the resource to generate a respective relevance score for each of the plurality of concept terms, wherein each of the respective relevance scores measures a predicted relevance of the corresponding concept term to the resource.

10. The system of claim 8 , wherein the numeric representations are vectors of floating point values.

11. The system of claim 8 , wherein the numeric representations are vectors of quantized integer values, and wherein an encoding of the quantized integer values represents floating point values.

12. The system of claim 8 , the operations further comprising:
obtaining the plurality of features of the resource in response to receiving an indication that an online advertisement auction is to be conducted to select one or more advertisements for inclusion in a particular presentation of the resource.

13. The system of claim 12 , the operations further comprising:
selecting one or more of the concept terms as advertising keywords to be used in selecting candidate advertisements for participation in the online advertisement auction based on the classification of the relevance of the concept terms to the resource.

14. The system of claim 8 , wherein a particular feature of the plurality of features is not discrete, and wherein the method further comprises:
hashing the particular feature to generate a hashed feature;
partitioning the hashed feature into a particular partition of a predetermined set of partitions; and
processing a value corresponding to the particular partition using the embedding function for the feature.

15. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
receiving an input comprising a plurality of features of a resource, wherein each of the features is of a different feature type;
generating an alternative representation of the features of the resource, comprising:
generating a respective numeric representation of each the features by processing each of the features using a respective embedding function, wherein each of the embedding functions is specific to features of a respective feature type, and
processing the respective numeric representations through one or more neural network layers to generate the alternative representation; and

providing the alternative representation of the features of the resource as input to a neural network classifier for classification of a relevance of a plurality of concept terms to the resource.

16. The computer storage media of claim 15 , wherein the neural network classifier is configured to:
process the alternative representation of the features of the resource to generate a respective relevance score for each of the plurality of concept terms, wherein each of the respective relevance scores measures a predicted relevance of the corresponding concept term to the resource.

17. The computer storage media of claim 15 , wherein the numeric representations are vectors of floating point values.

18. The computer storage media of claim 15 , wherein the numeric representations are vectors of quantized integer values, and wherein an encoding of the quantized integer values represents floating point values.

19. The computer storage media of claim 15 , the operations further comprising:
obtaining the plurality of features of the resource in response to receiving an indication that an online advertisement auction is to be conducted to select one or more advertisements for inclusion in a particular presentation of the resource.

20. The computer storage media of claim 19 , the operations further comprising:
selecting one or more of the concept terms as advertising keywords to be used in selecting candidate advertisements for participation in the online advert

Assignees

• Google Inc

Inventors

• Chen Kai
• Song Xiaodan
• Corrado Gregory S
• Zhang Kun
• Dean Jeffrey A
• Rabii Bahman

Classifications

• G06N3/042

  Knowledge-based neural networks; Logical representations of neural networks · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/09

  Supervised learning · CPC title