Systems and methods for controllable text summarization

Patent metadata

Field	Value
Publication number	US-11934781-B2
Application number	US-202017125468-A
Country	US
Kind code	B2
Filing date	Dec 17, 2020
Priority date	Aug 28, 2020
Publication date	Mar 19, 2024
Grant date	Mar 19, 2024

Abstract

Embodiments described herein provide a flexible controllable summarization system that allows users to control the generation of summaries without manually editing or writing the summary, e.g., without the user actually adding or deleting certain information under various granularity. Specifically, the summarization system performs controllable summarization through keywords manipulation. A neural network model is learned to generate summaries conditioned on both the keywords and source document so that at test time a user can interact with the neural network model through a keyword interface, potentially enabling multi-factor control.

First claim

What is claimed is:

1. A method of controllable text summarization of a document, the method comprising:
receiving, at a communication interface, an input text document;
extracting, via a language model that is trained by a training dataset of a plurality of documents and a plurality of corresponding keywords, one or more keywords by sequence labeling the one or more keywords from the input text document;
receiving, via a user interface, a control token sequence and one or more control parameters relating to a characteristic of a summary to be generated, wherein the characteristic of the summary includes any of: an entity mentioned in the input text document, a target length of the summary, and a type of the input text document, and wherein the one or more control parameters comprises a prompt corresponding to the type of the input text document;
modifying the one or more keywords based on the received control token sequence; and
generating, by the language model, the summary for the input text document based on the modified one or more keywords according to the one or more control parameters.

2. The method of claim 1 , wherein the prompt is selected from the group of: a first summary prefix that summarizes contributions of a research paper; a second summary prefix that summarizes invention purpose of a patent document; and a third summary prefix that summarizes the input text document in a guided question and answer format.

3. The method of claim 1 , further comprising:
generating a first set of modified keywords and a first control parameter from the received control token sequence;
generating, by the language model, a first version of the summary for the input text document based on the first set of modified keywords according to the first control parameter;
generating a second set of modified keywords and a second control parameter from the received control token sequence; and
generating, by the language model, a second version of the summary for the input text document based on the second set of modified keywords according to the second control parameter.

4. The method of claim 1 , wherein the language model is trained by:
prepending a keyword sequence to a training source document separated with a special token;
inputting the training source document with the keyword sequence to the language model;
generating, by the language model, an output summary; and
updating the language model by maximizing a conditional probability of an output summary conditioned on the training source document and the keyword sequence.

5. The method of claim 4 , further comprising:
randomly dropping a subset of keywords from the keyword sequence during training.

6. A system of controllable text summarization of a document, the system comprising:
a communication interface that receives an input text document;
a memory that stores a language model that is trained by a training dataset of a plurality of documents and a plurality of corresponding keywords; and
one or more hardware processors that:
extracts, via the language model, one or more keywords by sequence labeling the one or more keywords from the input text document;
receives, via the communication interface, a control token sequence and one or more control parameters relating to a characteristic of a summary to be generated, wherein the characteristic of the summary includes any of: an entity mentioned in the input text document, a target length of the summary, and a type of the input text document, and wherein the one or more control parameters comprises a prompt corresponding to the type of the input text document;
modifies the one or more keywords based on the received control token sequence; and
generates, by the language model, the summary for the input text document based on the modified one or more keywords according to the one or more control parameters.

7. The system of claim 6 , wherein the prompt is selected from the group of: a first summary prefix that summarizes contributions of a research paper; a second summary prefix that summarizes invention purpose of a patent document; and a third summary prefix that summarizes the input text document in a guided question and answer format.

8. The system of claim 6 , wherein the one or more hardware processors further:
generates a first set of modified keywords and a first control parameter from the received control token sequence;
generates, by the language model, a first version of the summary for the input text document based on the first set of modified keywords according to the first control parameter;
generates a second set of modified keywords and a second control parameter from the received control token sequence; and
generates, by the language model, a second version of the summary for the input text document based on the second set of modified keywords according to the second control parameter.

9. The system of claim 6 , wherein the language model is trained by:
prepending a keyword sequence to a training source document separated with a special token;
inputting the training source document with the keyword sequence to the language model;
generating, by the language model, an output summary; and
updating the language model by maximizing a conditional probability of an output summary conditioned on the training source document and the keyword sequence.

10. The system of claim 9 , wherein the one or more hardware processors further:
randomly drops a subset of keywords from the keyword sequence during training.

11. A non-transitory processor-readable medium storing a plurality of processor-executable instructions for controllable text summarization of a document, the instructions being executed by one or more processors to perform operations comprising:
receiving, at a communication interface, an input text document;
extracting, via a language model that is trained by a training dataset of a plurality of documents and a plurality of corresponding keywords, one or more keywords by sequence labeling the one or more keywords from the input text document;
receiving, via a user interface, a control token sequence and one or more control parameters relating to a characteristic of a summary to be generated, wherein the characteristic of the summary includes any of: an entity mentioned in the input text document, a target length of the summary, and a type of the input text document, and wherein the one or more control parameters comprises a prompt corresponding to the type of the input text document;
modifying the one or more keywords based on the received control token sequence; and
generating, by the language model, the summary for the input text document based on the modified one or more keywords according to the one or more control parameters.

12. The non-transitory processor-readable of claim 11 , wherein the prompt is selected from the group of: a first summary prefix that summarizes contributions of a research paper; a second summary prefix that summarizes invention purpose of a patent document; and a third summary prefix that summarizes the input text document in a guided question and answer format.

13. The non-transitory processor-readable of claim 11 , wherein the operations further comprise: generating a first set of modified keywords and a first control parameter from the received control token sequence; generating, by the language model, a first version of the summary for the input text document based on the first set of modified keywords according to the first control parameter; generating a second set of modified keywords and a second control parameter from the received control token sequence; and generatin

Assignees

• Salesforce Inc

Inventors

• He Junxian
• Kryscinski Wojciech
• Mccann Bryan

Classifications

• G06F40/284 Primary

  Lexical analysis, e.g. tokenisation or collocates · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06F3/0483

  Interaction with page-structured environments, e.g. book metaphor · CPC title

• G06F16/345 Primary

  Summarisation for human users · CPC title