Scalable and effective document summarization framework

Patent metadata

Field	Value
Publication number	US-10042924-B2
Application number	US-201615019646-A
Country	US
Kind code	B2
Filing date	Feb 9, 2016
Priority date	Feb 9, 2016
Publication date	Aug 7, 2018
Grant date	Aug 7, 2018

Abstract

Systems, methods, and apparatuses are disclosed for adaptively generating a summary of web-based content based on an attribute of a mobile communication device having transmitted a request for the web-based content. By adaptively generating the summary based on an attribute of the mobile communication device such as an amount of visual space available or a number of characters permitted in the interface, a display of the web-based content may be controlled on the mobile communication device in a way that was not previously available. This enables control of displaying web-based content that has been adaptively generated to be displayed on limited display screens based on a learned attribute of the mobile communication device requesting the web-based content.

First claim

What is claimed is:

1. A summarization engine comprising:
a memory configured to store a document including textual information;
an interface configured to receive a viewing request from a communication device, the viewing request corresponding to the document; and
a processor configured to:
communicate with the memory and interface;
in response to receiving the viewing request, determine a target summary length, wherein the target summary length identifies a targeted length for a generated summary;
extract the textual information from the document;
parse the textual information;
identify a plurality of sentence structures from the textual information based on the parsing;
assign each sentence structure a weighted score based on the target summary length;
generate, in accordance with a summarization policy, candidate summaries to include one or more sentence structures from the plurality of sentence structures;
determine a candidate score for each candidate summary as a linear function of the scores assigned to each sentence structure included in the respective candidate summary;
learn coefficients of the linear function from a training dataset of documents and authored summaries via a predetermined learning algorithm; and
select, from the generated candidate summaries, a generated summary determined to have the highest candidate score under the linear function.

2. The summarization engine of claim 1 , wherein the processor is configured to generate the weighted score by:
determining a position of each sentence structure within the document; and
generating the weighted score of each sentence structure based on the determined position of each corresponding sentence structure.

3. The summarization engine of claim 1 , wherein the processor is configured to generate the weighted score by:
determining a content of each sentence structure within the document; and
generating the weighted score of each sentence structure based on the determined content of each corresponding sentence structure.

4. The summarization engine of claim 1 , wherein the processor is configured to generate the weighted score by:
identifying one or more tokens from each sentence structure, wherein each identified token corresponds to a token type;
determining a token type of each identified token; and
generating the weighted score of each sentence structure based on the determined token type of each identified token of each corresponding sentence structure.

5. The summarization engine of claim 1 , wherein the processor is configured to generate the weighted score by:
identifying one or more lexical cues from each sentence structure; and
generating the weighted score of each sentence structure based on the identified lexical cues of each corresponding sentence structure.

6. The summarization engine of claim 1 , wherein the target summary length is included in the viewing request; and
wherein the processor is configured to determine the target summary length by extracting the target summary length from the viewing request.

7. The summarization engine of claim 6 , wherein the target summary length corresponds to an attribute of the communication device transmitting the viewing request.

8. A method for generating a summary of a document, the method comprising:
receiving, through an interface, a viewing request from a communication device, the viewing request corresponding to a document including textual information stored on a memory;
in response to receiving the viewing request, determining a target summary length;
extracting the textual information from the document;
parsing the textual information;
identifying a plurality of sentence structures from the textual information based on the parsing;
assigning each sentence structure a weighted score based on the target summary length;
generating, in accordance with a summarization policy, candidate summaries to include one or more sentence structures from the plurality of sentence structures;
determining a candidate score for each candidate summary as a linear function of the scores assigned to each sentence structure included in the respective candidate summary;
learning coefficients of the linear function from a training dataset of documents and authored summaries via a predetermined learning algorithm; and
selecting, from the generated candidate summaries, a generated summary determined to have the highest candidate score under the linear function.

9. The method of claim 8 , further comprising generating the weighted score by:
determining a position of each sentence structure within the document; and
generating the weighted score of each sentence structure based on the determined position of each corresponding sentence structure.

10. The method of claim 8 , further comprising generating the weighted score by:
determining a content of each sentence structure within the document; and
generating the weighted score of each sentence structure based on the determined content of each corresponding sentence structure.

11. The method of claim 8 , further comprising generating the weighted score by:
identifying one or more tokens from each sentence structure, wherein each identified token corresponds to a token type;
determining a token type of each identified token; and
generating the weighted score of each sentence structure based on the determined token type of each identified token of each corresponding sentence structure.

12. The method of claim 8 , wherein assigning each sentence structure the weighted score based on the analysis and the target summary length comprises:
scoring each sentence structure as it appears in the document; and
scoring each sentence structure as it appears in a partial summary including one or more sentence structures already selected from the document.

13. The method of claim 8 , wherein the predetermined learning algorithm is a structured perception.

14. The method of claim 8 , wherein generating, in accordance with the summarization policy, the candidate summaries comprises:
considering each sentence structure included in the document;
revising each sentence score corresponding to a considered sentence structure; and
adding a highest-scoring sentence structure to the candidate summary such that the target summary length is not exceeded.

15. The method of claim 8 , wherein generating, in accordance with the summarization policy, the candidate summaries comprises:
considering each sentence structure in the document that appears after every sentence structure in the partial summary;
revising each sentence score corresponding to a considered sentence structure; and
adding a highest-scoring sentence structure to the candidate summary such that the target summary length is not exceeded.

16. The method of claim 8 , wherein generating, in accordance with the summarization policy, the candidate summaries comprises:
considering each sentence structure in the document that appears at a beginning or an end of a paragraph in the document;
revising each sentence score corresponding to a considered sentence structure; and
adding a highest-scoring sentence to the candidate summary such that the target summary length is not exceeded.

17. A method for generating a summary of a document, the method comprising:
receiving, through an interface, a viewing request from a communication device, the viewing request corresponding to a document including textual information stored on a memory;
in response to receiving the viewing request, determining a target summary length;
ext

Assignees

• Yahoo Holdings Inc
• Oath Inc

Inventors

• Billawal Youssef
• Mehdad Yashar
• Radev Dragomir
• Stent Amanda
• Thadani Kapil

Classifications

• G06F40/35

  Discourse or dialogue representation · CPC title

• G06F40/211

  Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars · CPC title

• G06F40/284

  Lexical analysis, e.g. tokenisation or collocates · CPC title

• G06F16/345 Primary

  Summarisation for human users · CPC title

• G06F17/30719 Primary

  Physics · mapped topic