Generative text summarization system and method

Patent metadata

Field	Value
Publication number	US-11562144-B2
Application number	US-202016819655-A
Country	US
Kind code	B2
Filing date	Mar 16, 2020
Priority date	Mar 16, 2020
Publication date	Jan 24, 2023
Grant date	Jan 24, 2023

Abstract

A generative automatic text summarization system and method is disclosed that may adopt a search and reranking strategy to improve the performance of a summarization task. The system and method may employ a transformer neural model to assist with the summarization task. The transformer neural model may be trained to learn human abstracts and may then be operable to generate abstractive summaries. With multiple summary hypothesis generated, a best-first search algorithm and reranking algorithm may be employed to select the best candidate summary as part of the output summary.

First claim

What is claimed is:

1. A method for a generative text summarization model, comprising:
receiving an input text dataset;
enlarging a search space for one or more candidate words to be selected for inclusion in a text summary, wherein the one or more candidate words included within the search space are ranked using a best-first search algorithm; and
re-ranking the one or more candidate words to be included in the text summary using a soft-bound word-reward (SBWR) algorithm, wherein the SBWR algorithm applies a diminishing reward value to the one or more candidate words when the text summary exceeds a predicated length threshold, and wherein the SBWR algorithm applies an increased reward value to the one or more candidate words when the text summary is below the predicated length threshold.

2. The method of claim 1 , wherein the SBWR algorithm selects the one or more candidate words when the text summary is equivalent to the predicated length threshold.

3. The method of claim 1 , wherein the SBWR algorithm operates using the following formula:

S
ˆ

sbwr

(

x
,
y

)

=

S
⁡
(

x
,
y

)

+

r
⁢

∑

i
=
1

❘
"\[LeftBracketingBar]"

y

❘
"\[RightBracketingBar]"

σ
⁡
(

ℒ
pred

-
i

)

.

4. The method of claim 1 further comprising: smoothing the diminishing reward value and the increased reward value using a sigmoid function.

5. The method of claim 1 further comprising: scaling the diminishing reward value and the increased reward value using a value that is trained to select the one or more candidate words to be included in the text summary.

6. The method of claim 1 , re-ranking the one or more candidate words when the input text dataset exceeds a predefined length threshold.

7. The method of claim 1 further comprising: calculating a BP normalization that applies a penalty to the one or more candidate words that do not meet the predicated length threshold.

8. The method of claim 7 , wherein the BP normalization is calculated by adding a logarithmic value of a brevity penalty with a length normalization scoring function.

9. The method of claim 8 , wherein the brevity penalty is designed so that the generative text summarization model does not produce a short translation from the input text dataset.

10. The method of claim 8 , wherein the brevity penalty includes a copy rate value that reduces the brevity penalty toward zero.

11. The method of claim 1 further comprising: training the generative text summarization model using a transformer neural model.

12. The method of claim 11 , wherein the transformer neural model includes an encoder machine learning algorithm and a decoder machine learning algorithm.

13. The method of claim 12 further comprising: inputting the input text dataset to the encoder machine learning algorithm; and inputting a target summary text dataset to the decoder machine learning algorithm.

14. The method of claim 13 , wherein the transformer neural model determines a probability value for one or more target summary tokens using one or more source tokens.

15. The method of claim 14 , wherein the transformer neural model determines the probability value for the one or more target summary tokens using the one or more source tokens based on the following equation:

P
⁡
(

y
|
x

)

=

∏

j
=
1

❘
"\[LeftBracketingBar]"

y

❘
"\[RightBracketingBar]"

P
⁡
(

y
j

|

y

<
j

,
x

)

.

16. A system operable to employ a generative text summarization model, comprising:
a memory operable to store an input text dataset;
a processor operable to:
enlarge a search space for one or more candidate words to be selected for inclusion in a text summary, wherein the one or more candidate words included within the search space are ranked using a best-first search algorithm; and
re-rank the one or more candidate words to be included in the text summary using a soft-bound word-reward (SBWR) algorithm, wherein the SBWR algorithm applies a diminishing reward value to the one or more candidate words when the text summary exceeds a predicated length threshold, and wherein the SBWR algorithm applies an increased reward value to the one or more candidate words when the text summary is below the predicated length threshold.

17. The system of claim 16 , wherein the SBWR algorithm selects the one or more candidate words when the text summary is equivalent to the predicated length threshold.

18. The system of claim 16 , wherein the processor is further operable to: smooth the diminishing reward value and the increased reward value using a s

Assignees

• Bosch Gmbh Robert

Inventors

• Song Kaiqiang
• Wang Bingqing

Classifications

• G06N3/0454

  Physics · mapped topic

• G06F16/24578

  using ranking · CPC title

• G06N3/088

  Non-supervised learning, e.g. competitive learning · CPC title

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06F40/56 Primary

  Natural language generation · CPC title