Language independent representations

Patent metadata

Field	Value
Publication number	US-9990361-B2
Application number	US-201514878794-A
Country	US
Kind code	B2
Filing date	Oct 8, 2015
Priority date	Oct 8, 2015
Publication date	Jun 5, 2018
Grant date	Jun 5, 2018

Abstract

Snippets can be represented in a language-independent semantic manner. Each portion of a snippet can be represented by a combination of a semantic representation and a syntactic representation, each in its own dimensional space. A snippet can be divided into portions by constructing a dependency structure based on relationships between words and phrases. Leaf nodes of the dependency structure can be assigned: A) a semantic representation according to pre-defined word mappings and B) a syntactic representation according to the grammatical use of the word. A trained semantic model can assign to each non-leaf node of the dependency structure a semantic representation based on a combination of the semantic and syntactic representations of the corresponding lower-level nodes. A trained syntactic model can assign to each non-leaf node a syntactic representation based on a combination of the syntactic representations of the corresponding lower-level nodes and the semantic representation of that node.

First claim

We claim:

1. A method for generating improved machine translations using a language independent representation of a snippet that accounts for semantics of the snippet, the method comprising:
receiving the snippet, wherein the snippet comprises a digital representation of one or more words or groups of characters from a first natural language;
building a dependency structure based on the received snippet, the dependency structure comprising multiple nodes;
obtaining semantic and syntactic representations of leaf nodes of the dependency structure;
generating a first semantic representation corresponding to a selected non-leaf node of the dependency structure by applying a semantic model to semantic and syntactic representations of parent nodes of the selected non-leaf node;
mapping the first semantic representation into a virtual language independent space;
locating a second semantic representation that is the closest, in the virtual language independent space, to the first semantic representation; and
providing, as a translation of the snippet, content in a second natural language that was a basis for creating the second semantic representation.

2. The method of claim 1 wherein:
the syntactic representations are syntactic vectors; and
the semantic representations are semantic vectors.

3. The method of claim 2 wherein the semantic model comprises:
a tensor function that generates a tensor based on two syntactic vectors;
a first matrix function that generates a first matrix based on two syntactic vectors;
a second matrix function that generates a second matrix based on two syntactic vectors; and
an offset function that generates an offset vector based on two syntactic vectors.

4. The method of claim 3 wherein generating the semantic representation corresponding to the selected non-leaf node of the dependency structure comprises:
obtaining a first syntactic vector corresponding to a first parent node of the selected non-leaf node;
obtaining a second syntactic vector corresponding to a second parent node of the selected non-leaf node;
generating the tensor by applying the tensor function to the first syntactic vector and the second syntactic vector;
generating the first matrix by applying the first matrix function to the first syntactic vector and the second syntactic vector;
generating the second matrix by applying the second matrix function to the first syntactic vector and the second syntactic vector; and
generating the offset vector by applying the offset function to the first syntactic vector and the second syntactic vector.

5. The method of claim 4 wherein generating the semantic representation corresponding to the selected non-leaf node of the dependency structure comprises:
obtaining a first semantic vector corresponding to the first parent node of the selected non-leaf node;
obtaining a second semantic vector corresponding to the second parent node of the selected non-leaf node;
computing a first result by multiplying together: the tensor, the first semantic vector, and the second semantic vector;
computing a second result by multiplying together: the first matrix with the first semantic vector;
computing a third result by multiplying together: the second matrix with the second semantic vector; and
computing the semantic representation corresponding to the selected non-leaf node as the sum of: the first result, the second result, the third result, and the offset vector.

6. The method of claim 2 further comprising:
generating a syntactic representation corresponding to the selected non-leaf node of the dependency structure by applying a syntactic model to syntactic representations of the parent nodes of the selected non-leaf node and to the semantic representation corresponding to the selected non-leaf node.

7. The method of claim 6 wherein the syntactic model comprises:
a tensor function that generates a tensor based on two syntactic vectors;
a first matrix function that generates a first matrix based on two syntactic vectors;
a second matrix function that generates a second matrix based on two syntactic vectors;
an offset function that generates an offset vector based on two syntactic vectors; and
a mapping matrix that is a linear mapping from semantic space to syntactic space.

8. The method of claim 7 wherein generating the syntactic representation corresponding to the selected non-leaf node comprises:
obtaining a first syntactic vector corresponding to a first parent node of the selected non-leaf node;
obtaining a second syntactic vector corresponding to a second parent node of the selected non-leaf node;
generating the tensor by applying the tensor function to the first syntactic vector and the second syntactic vector;
generating the first matrix by applying the first matrix function to the first syntactic vector and the second syntactic vector;
generating the second matrix by applying the second matrix function to the first syntactic vector and the second syntactic vector; and
generating the offset vector by applying the offset function to the first syntactic vector and the second syntactic vector.

9. The method of claim 8 wherein generating the syntactic representation corresponding to the selected non-leaf node of the dependency structure comprises:
computing a first result by multiplying together: the first matrix with a first semantic vector;
computing a second result by multiplying together: the second matrix with a second semantic vector;
computing a third result by multiplying together: the mapping matrix with the semantic representation corresponding to the selected non-leaf node; and
computing the syntactic representation corresponding to the selected non-leaf node as the sum of: the first result, the second result, the third result, the tensor, and the offset vector.

10. The method of claim 1 wherein the dependency structure is a binary tree structure.

11. The method of claim 1 wherein the selected non-leaf node of the dependency structure is the root node of the dependency structure, and wherein the method further comprises:
generating, for a composition that includes the semantic representation corresponding to the selected non-leaf node of the dependency structure, a score; and
adjusting parameters of the semantic model based on the score.

12. The method of claim 11 wherein generating the score comprises:
applying, to the semantic representation corresponding the root node and to a syntactic representation corresponding the root node, a scoring neural network that is trained to receive a semantic vector and a syntactic vector and generate the score indicating how reliably the semantic vector maps into the language independent space.

13. The method of claim 12 wherein generating the score further comprises:
applying the scoring neural network to multiple nodes of the dependency structure to compute corresponding scores for the multiple nodes of the dependency structure; and
combining, as the score for the composition, the scores for the multiple nodes of the dependency structure.

14. The method of claim 13 wherein combining the scores for the multiple nodes of the dependency structure comprises:
summing the scores for the multiple nodes for the dependency structure; or
multiplying each selected score of the multiple nodes for the dependency structure by (½)^depth, wherein the depth is the maximum number of edges between the node corresponding to that selected score and the root node of the dependency structure, and summing the results of the multiplications.

15. The method of claim 1 wherein:
at least tw

Assignees

• Facebook Inc

Inventors

• Zhang Ying
• Huang Fei
• Wang Xiaolong

Classifications

• G06F40/42

  Data-driven translation · CPC title

• G06F40/58 Primary

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title

• G06F40/211

  Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars · CPC title

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06F17/2785

  Physics · mapped topic