Abstraction of text summarization

Patent metadata

Field	Value
Publication number	US-10909157-B2
Application number	US-201816051188-A
Country	US
Kind code	B2
Filing date	Jul 31, 2018
Priority date	May 22, 2018
Publication date	Feb 2, 2021
Grant date	Feb 2, 2021

Abstract

A system is disclosed for providing an abstractive summary of a source textual document. The system includes an encoder, a decoder, and a fusion layer. The encoder is capable of generating an encoding for the source textual document. The decoder is separated into a contextual model and a language model. The contextual model is capable of extracting words from the source textual document using the encoding. The language model is capable of generating vectors paraphrasing the source textual document based on pre-training with a training dataset. The fusion layer is capable of generating the abstractive summary of the source textual document from the extracted words and the generated vectors for paraphrasing. In some embodiments, the system utilizes a novelty metric to encourage the generation of novel phrases for inclusion in the abstractive summary.

First claim

What is claimed is:

1. A system for providing an abstractive summary of a source textual document, the system comprising:
an encoder generating, using a processor, an encoding for the source textual document and encoder temporal attention context vectors;
a decoder separated into a contextual model and a language model, wherein the contextual model is generating, using the processor, decoder intra-attention context vectors and first vectors for extracting words from the source textual document using the encoding, and the language model is generating, using the processor, second vectors for composing paraphrases of selected words in the source textual document from a fixed vocabulary;
a reference vector combining the encoder temporal attention context vectors, the decoder intra-attention context vectors, and the first vectors; and
a fusion layer generating the abstractive summary of the source textual document from the reference vector and the second vectors.

2. The system of claim 1 , wherein the system comprises a neural network.

3. The system of claim 1 , wherein the system utilizes a novelty metric to encourage generation of novel phrases for inclusion in the abstractive summary.

4. The system of claim 1 , wherein the system utilizes a metric that measures word-overlap of the abstractive summary with a ground-truth summary for the source textual document.

5. The system of claim 1 , further comprising:
prior to the language model of the decoder generating the second vectors, training the language model using a CNN/Daily Mail dataset.

6. The system of claim 1 , wherein the language model comprises a multi-layer unidirectional Long Short-Term Memory (LSTM) unit for generating vectors.

7. A method for providing an abstractive summary of a source textual document, the method performed on a computer and comprising:
generating, using a processor, at an encoder an encoding for the source textual document and an encoder temporal attention context vectors;
generating, using a contextual model of a decoder and the encodings, a decoder intra-attention context vectors and first vectors for extracting words from the source textual document;
generating, using a language model of the decoder that is separate from the contextual model, second vectors for composing paraphrases of selected words in the source textual document from a fixed vocabulary;
generating a reference vector composed of the encoder temporal attention context vectors, the decoder intra-attention context vectors, and the first vectors; and
generating the abstractive summary of the source textual document from the reference vector and the second vectors.

8. The method of claim 7 , further comprising utilizing a novelty metric to encourage generation of novel phrases for inclusion in the abstractive summary.

9. The method of claim 7 , further comprising utilizing a metric that measures word-overlap of the abstractive summary with a ground-truth summary for the source textual document.

10. The method of claim 7 , further comprising:
prior to the language model of the decoder generating the second vectors, training the language model using a CNN/Daily Mail dataset.

11. The method of claim 7 , wherein the abstractive summary comprises at least one word from the fixed vocabulary.

12. A non-transitory machine readable medium having stored thereon instructions for performing a method comprising machine executable code which when executed by at least one machine, causes the machine to:
generate, at an encoder, an encoding and an encoder temporal attention vectors for a source textual document;
generate, using a contextual model of a decoder and the encodings, a decoder intra-attention context vectors and first vectors for extracting words from the source textual document;
generate, using a language model of the decoder that is separate from the contextual model, second vectors for composing paraphrases of selected words in the source textual document from a fixed vocabulary;
generate a reference vector composed of the encoder temporal attention vectors, the decoder intra-attention context vectors, and the first vectors; and
generate an abstractive summary of the source textual document from the reference vector and the second vectors.

13. The non-transitory machine readable medium of claim 12 , further storing instructions which when executed by the at least one machine cause the machine to utilize a novelty metric to encourage generation of novel phrases for inclusion in the abstractive summary.

14. The non-transitory machine readable medium of claim 12 , further storing instructions which when executed by the at least one machine cause the machine to utilize a metric that measures word-overlap of the abstractive summary with a ground-truth summary for the source textual document.

15. The non-transitory machine readable medium of claim 12 , storing instructions which when executed by the at least one machine, cause the machine to train the language model of the decoder using a CNN/Daily Mail dataset.

16. The non-transitory machine readable medium of claim 12 , wherein the abstractive summary includes at least one word from the fixed vocabulary.

Assignees

• Salesforce Com Inc

Inventors

• Paulus Romain
• Kryscinski Wojciech
• Xiong Caiming

Classifications

• G06F18/253

  of extracted features · CPC title

• G06F18/24143

  Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN] · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title