Generating distributed word embeddings using structured information

Patent metadata

Field	Value
Publication number	US-9922025-B2
Application number	US-201715671303-A
Country	US
Kind code	B2
Filing date	Aug 8, 2017
Priority date	May 8, 2015
Publication date	Mar 20, 2018
Grant date	Mar 20, 2018

Abstract

A computer program that generates a vector representation of a set of natural language text in a natural language processing system by: (i) receiving a first set of natural language text and a set of information pertaining to the first set of natural language text, where the information includes a dependency parse tree including a root node and a plurality of nodes that depend from the root node, where the root node represents the first set of natural language text, and where the plurality of nodes that depend from the root node represent context features of the first set of natural language text; and (ii) generating, by the natural language processing system, a first vector representation of the first set of natural language text, wherein the generating includes adding vector representations for the context features represented by the plurality of nodes that depend from the root node.

First claim

What is claimed is:

1. A method for generating a vector representation of a set of natural language text in a natural language processing system, the method comprising:
receiving, by the natural language processing system, a first set of natural language text and a set of information pertaining to the first set of natural language text, where the information includes a dependency parse tree including a root node and a plurality of nodes that depend from the root node, where the root node represents the first set of natural language text, and where the plurality of nodes that depend from the root node represent context features of the first set of natural language text;
generating, by the natural language processing system, a first vector representation of the first set of natural language text, wherein the generating includes adding vector representations for the context features represented by the plurality of nodes that depend from the root node; and
comparing, by the natural language processing system, the generated first vector representation to a second vector representation to determine, in the natural language processing system, an amount of similarity between the first set of natural language text and a second set of natural language text represented by the second vector representation.

2. The method of claim 1 , wherein:
the first set of natural language text is part of an input sentence; and
the context features of the first set of natural language text represented by the plurality of nodes that depend from the root node correspond to words or phrases from the input sentence other than the first set of natural language text.

3. The method of claim 2 , wherein:
the context features of the first set of natural language text represented by the plurality of nodes that depend from the root node include: (i) the respective words or phrases to which the context features correspond, and (ii) contextual information indicating a relationship between the respective words or phrases and the first set of natural language text.

4. The method of claim 3 , wherein:
the first set of natural language text is a verb.

5. The method of claim 4 , wherein:
a first word or phrase corresponding to and included in a first context feature of the first set of natural language text is a subject of the verb; and
the contextual information included in the first context feature indicates that the first word or phrase is a subject of the verb.

6. The method of claim 4 , wherein:
a first word or phrase corresponding to and included in a first context feature of the first set of natural language text is an object of the verb; and
the contextual information included in the first context feature indicates that the first word or phrase is an object of the verb.

7. The method of claim 4 , wherein:
a first word or phrase corresponding to and included in a first context feature of the first set of natural language text is a prepositional phrase that modifies the verb; and
the contextual information included in the first context feature indicates that the first word or phrase is a prepositional phrase that modifies the verb.

8. A computer program product for generating a vector representation of a set of natural language text in a natural language processing system, the computer program product comprising a computer readable storage medium having stored thereon:
program instructions to receive, by the natural language processing system, a first set of natural language text and a set of information pertaining to the first set of natural language text, where the information includes a dependency parse tree including a root node and a plurality of nodes that depend from the root node, where the root node represents the first set of natural language text, and where the plurality of nodes that depend from the root node represent context features of the first set of natural language text;
program instructions to generate, by the natural language processing system, a first vector representation of the first set of natural language text, wherein the generating includes adding vector representations for the context features represented by the plurality of nodes that depend from the root node; and
program instructions to compare, by the natural language processing system, the generated first vector representation to a second vector representation to determine, in the natural language processing system, an amount of similarity between the first set of natural language text and a second set of natural language text represented by the second vector representation.

9. The computer program product of claim 8 , wherein:
the first set of natural language text is part of an input sentence; and
the context features of the first set of natural language text represented by the plurality of nodes that depend from the root node correspond to words or phrases from the input sentence other than the first set of natural language text.

10. The computer program product of claim 9 , wherein:
the context features of the first set of natural language text represented by the plurality of nodes that depend from the root node include: (i) the respective words or phrases to which the context features correspond, and (ii) contextual information indicating a relationship between the respective words or phrases and the first set of natural language text.

11. The computer program product of claim 10 , wherein:
the first set of natural language text is a verb.

12. The computer program product of claim 11 , wherein:
a first word or phrase corresponding to and included in a first context feature of the first set of natural language text is a subject of the verb; and
the contextual information included in the first context feature indicates that the first word or phrase is a subject of the verb.

13. The computer program product of claim 11 , wherein:
a first word or phrase corresponding to and included in a first context feature of the first set of natural language text is an object of the verb; and
the contextual information included in the first context feature indicates that the first word or phrase is an object of the verb.

14. The computer program product of claim 11 , wherein:
a first word or phrase corresponding to and included in a first context feature of the first set of natural language text is a prepositional phrase that modifies the verb; and
the contextual information included in the first context feature indicates that the first word or phrase is a prepositional phrase that modifies the verb.

15. A computer system for generating a vector representation of a set of natural language text in a natural language processing system, the computer system comprising:
a processor(s) set; and
a computer readable storage medium;
wherein:
the processor set is structured, located, connected and/or programmed to run program instructions stored on the computer readable storage medium; and
the program instructions include:
program instructions to receive, by the natural language processing system, a first set of natural language text and a set of information pertaining to the first set of natural language text, where the information includes a dependency parse tree including a root node and a plurality of nodes that depend from the root node, where the root node represents the first set of natural language text, and where the plurality of nodes that depend from the root node represent context features of the first set of natural language text;
program instructions to generate, by the natural language processing system, a first vector representation

Assignees

• IBM

Inventors

• Cross Iii James H
• Fan James J
• Xiang Bing
• Zhou Bowen

Classifications

• G06F16/36 Primary

  Creation of semantic tools, e.g. ontology or thesauri · CPC title

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06F40/211

  Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars · CPC title

• G06F17/2785 Primary

  Physics · mapped topic

• G06F17/30731

  Physics · mapped topic