Relation extraction from text using machine learning

Patent metadata

Field	Value
Publication number	US-11625573-B2
Application number	US-201816173534-A
Country	US
Kind code	B2
Filing date	Oct 29, 2018
Priority date	Oct 29, 2018
Publication date	Apr 11, 2023
Grant date	Apr 11, 2023

Abstract

A first neural network is operated on a processor and a memory to encode a first natural language string into a first sentence encoding including a set of word encodings. Using a word-based attention mechanism with a context vector, a weight value for a word encoding within the first sentence encoding is adjusted to form an adjusted first sentence encoding. Using a sentence-based attention mechanism, a first relationship encoding corresponding to the adjusted first sentence encoding is determined. An absolute difference between the first relationship encoding and a second relationship encoding is computed. Using a multi-layer perceptron, a degree of analogical similarity between the first relationship encoding and a second relationship encoding is determined.

First claim

What is claimed is:

1. A method comprising:
operating a first neural network on a processor and a memory to encode a first natural language string into a first sentence encoding comprising a set of word encodings;
adjusting, using a word-based attention mechanism with a context vector, a weight value for a word encoding within the first sentence encoding to form an adjusted first sentence encoding, a sentence layer of the first neural network comprising the word-based attention mechanism with the context vector, the word-based attention mechanism trained to form the adjusted first sentence encoding by adjusting a plurality of weights within the sentence layer;
generating, using a sentence-based attention mechanism, a first relationship encoding corresponding to the adjusted first sentence encoding, the sentence-based attention mechanism further adjusting a plurality of adjusted sentence encodings output from the word-based attention mechanism, the first relationship encoding comprising a vector representation of a relationship between entities, the relationship expressed in the first natural language string, a relation layer of the first neural network comprising the sentence-based attention mechanism, the sentence-based attention mechanism trained to generate the first relationship encoding by adjusting a plurality of weights within the relation layer;
computing an absolute difference between the first relationship encoding and a second relationship encoding; and
determining, using a multi-layer perceptron, a degree of analogical similarity between the first relationship encoding and the second relationship encoding.

2. The method of claim 1 , further comprising:
operating a second neural network on a processor and a memory to encode a second natural language string into a second sentence encoding comprising a second set of word encodings;
adjusting, using a second word-based attention mechanism with a second context vector, a weight value for a word encoding within the second sentence encoding to form an adjusted second sentence encoding; and
determining, using a second sentence-based attention mechanism, the second relationship encoding corresponding to the adjusted second sentence encoding.

3. The method of claim 2 , wherein the first neural network and the second neural network are identically structured.

4. The method of claim 2 , wherein the word-based attention mechanism with the context vector and the second word-based attention mechanism with the second context vector are identically structured.

5. The method of claim 2 , wherein the sentence-based attention mechanism and the second sentence-based attention mechanism are identically structured.

6. The method of claim 2 , further comprising:
determining, using an output unit including a sigmoid activation function, that the first relationship encoding and the second relationship encoding correspond to an analogous relationship.

7. The method of claim 2 , further comprising:
determining, using an output unit including a sigmoid activation function, that the first relationship encoding and the second relationship encoding do not correspond to an analogous relationship.

8. The method of claim 2 , further comprising:
training, using a set of pairs of natural language strings, wherein each natural language string in the set of pairs of natural language strings expresses a relationship between entities included in the natural language string, the first neural network and the second neural network.

9. The method of claim 8 , further comprising:
generating a set of relation pairs, wherein each relation pair in the set of relation pairs comprises a pair of entities and a relationship relating the pair of entities;
generating a set of positive example pairs, wherein each positive example pair comprises two relation pairs, a relationship of each relation pair in the set of positive example pairs being equivalent to each other;
generating a set of negative example pairs, wherein each negative example pair comprises two relation pairs, a relationship of each relation pair in the set of negative example pairs not being equivalent to each other;
combining, forming a training set of example pairs, the set of positive example pairs and the set of negative example pairs; and
converting, by extracting from a text corpus a natural language string expressing a relationship between entities included in the natural language string, the training set of example pairs to a training set of pairs of natural language strings.

10. A computer usable program product comprising one or more computer-readable storage media, and program instructions stored on at least one of the one or more computer-readable storage media, the stored program instructions comprising:
program instructions to operate a first neural network on a processor and a memory to encode a first natural language string into a first sentence encoding comprising a set of word encodings;
program instructions to adjust, using a word-based attention mechanism with a context vector, a weight value for a word encoding within the first sentence encoding to form an adjusted first sentence encoding, a sentence layer of the first neural network comprising the word-based attention mechanism with the context vector, the word-based attention mechanism trained to form the adjusted first sentence encoding by adjusting a plurality of weights within the sentence layer;
program instructions to generate, using a sentence-based attention mechanism, a first relationship encoding corresponding to the adjusted first sentence encoding, the sentence-based attention mechanism further adjusting a plurality of adjusted sentence encodings output from the word-based attention mechanism, the first relationship encoding comprising a vector representation of a relationship between entities, the relationship expressed in the first natural language string, a relation layer of the first neural network comprising the sentence-based attention mechanism, the sentence-based attention mechanism trained to generate the first relationship encoding by adjusting a plurality of weights within the relation layer;
program instructions to compute an absolute difference between the first relationship encoding and a second relationship encoding; and
program instructions to determine, using a multi-layer perceptron, a degree of analogical similarity between the first relationship encoding and the second relationship encoding.

11. The computer usable program product of claim 10 , further comprising:
program instructions to operate a second neural network on a processor and a memory to encode a second natural language string into a second sentence encoding comprising a second set of word encodings;
program instructions to adjust, using a second word-based attention mechanism with a second context vector, a weight value for a word encoding within the second sentence encoding to form an adjusted second sentence encoding; and
program instructions to determine, using a second sentence-based attention mechanism, the second relationship encoding corresponding to the adjusted second sentence encoding.

12. The computer usable program product of claim 11 , wherein the first neural network and the second neural network are identically structured.

13. The computer usable program product of claim 11 , wherein the word-based attention mechanism with the context vector and the second word-based attention mechanism with the second context vector are identically structured.

14. The computer usable program product of claim 11 , wherein the sentence-based attention mechanism and the second s

Assignees

• IBM

Inventors

• Gliozzo Alfio Massimiliano
• Rossiello Gaetano
• Farrell Robert G

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/08

  Learning methods · CPC title