Systems and methods for determining structured proceeding outcomes

Patent metadata

Field	Value
Publication number	US-11568503-B2
Application number	US-201916446423-A
Country	US
Kind code	B2
Filing date	Jun 19, 2019
Priority date	Jun 19, 2018
Publication date	Jan 31, 2023
Grant date	Jan 31, 2023

Abstract

The present disclosure relates to systems and methods for analyzing and extracting data related to a structured proceeding, and for identifying, based on the analysis, at least one outcome associated with the structured proceeding. Embodiments provide for receiving data associated with a structured proceeding involving at least one party, the data including at least one docket entry, and analyzing, by an outcome location detector, the data to identify one or more docket entries in the at least one docket entry that are likely to include evidence of an outcome. Embodiments further include analyzing, by an outcome detector, the one or more docket entries determined to be likely to include evidence of an outcome to determine outcomes. The outcomes include at least one of a final outcome and at least one party outcome. The final outcome is associated with the structured proceeding overall, and the at least one party outcome is associated with a party of the at least one party that may have been terminated early.

First claim

The invention claimed is:

1. A method comprising:
receiving data associated with a structured proceeding involving at least one party, the data including at least one docket entry;
analyzing, by an outcome location detector, the data to identify one or more docket entries in the at least one docket entry that include evidence of an outcome; and
analyzing, by an outcome detector, the one or more docket entries to determine a final outcome associated with the structured proceeding overall and at least one party outcome associated with a party to a proceeding corresponding to the one or more docket entries, wherein analyzing the one or more docket entries comprises applying, by a deep learning classifier of the outcome detector, a hierarchical attention network approach to the one or more docket entries, wherein applying the hierarchical attention network approach comprises:
providing each word token of the one or more docket entries to a first word-level recurrent neural network (RNN),
providing an output of each corresponding first word-level RNN to a word-level attention layer configured to encode the one or more docket entries,
providing the encoded one or more docket entries to a second word-level RNN, and
providing an output of the second word-level RNN to a docket entry-level attention layer configured to encode a docket; and

generating, by the outcome location detector, a final outcome location prediction based on the encoded docket;
generating, by an outcome prediction detector, one or more docket entry outcome predictions corresponding to the one or more docket entries based on the encoded docket and the output of the second word-level RNN;
and
generating, by a final outcome detector, a final outcome based on the final outcome location prediction and the one or more docket entry outcome predictions.

2. The method of claim 1 , wherein the analyzing, by the outcome detector, the one or more docket entries further includes:
analyzing, by a docket entry classifier of the outcome detector, the one or more docket entries individually;
determining whether individual docket entries include an individual outcome associated with a respective individual docket entry; and
identifying the individual outcome associated with a respective individual docket entry, wherein the identifying the individual outcome associated with a respective individual docket entry is performed without considering other docket entries of the at least one docket entry.

3. The method of claim 1 , wherein the analyzing, by the outcome detector, the one or more docket entries further includes:
determining whether individual docket entries of the one or more docket entries include an individual outcome associated with a respective individual docket entry; and
identifying the final outcome based on the individual docket entry and at least one other individual outcome identified in another individual docket entry of the one or more docket entries.

4. The method of claim 1 , wherein the analyzing, by the outcome detector, the one or more docket entries further includes feeding features from the one or more docket entry outcome predictions to the final outcome detector.

5. The method of claim 1 , wherein the analyzing, by the outcome detector, the one or more docket entries further includes:
determining, by an party outcome detector, that the party was terminated prior to the final outcome of the structured proceeding;
calculating a probability that the determination that the party was terminated prior to the final outcome is accurate;
determining at least one docket entry associated with the termination of the party;
determining a docket entry outcome included in the associated at least one docket entry; and
assigning a party outcome to the party based on the at least one entry.

6. The method of claim 5 , wherein the determining the at least one docket entry associated with the termination of the party includes:
determining a termination date of the party, the termination date indicating a date when the party was terminated from the structured proceeding; and
identifying the at least one docket entry associated with the termination of the party based on the termination date of the party.

7. The method of claim 1 , further including pre-processing the data, wherein pre-processing includes:
extracting metadata from the data;
filtering the data, including identifying invalid dockets to be removed from the data;
annotating masked entities, including identifying entities to be normalized and annotating the identified entities for subsequent normalization;
identifying outcome-eligible parties; and
applying rules to identify particular events in the docket entries and associate the particular events with particular parties of the at least one party.

8. The method of claim 1 , further including storing the final outcome and the at least one party outcome in a database for subsequent retrieval.

9. A system comprising:
an outcome location detector configured to:
receive data associated with a structured proceeding involving at least one party, the data including at least one docket entry,
analyze the data to identify one or more docket entries in the at least one docket entry that include evidence of an outcome, and
generate a final outcome location prediction based on an encoded docket;

an outcome detector configured to:
analyze the one or more docket entries to determine a final outcome associated with the structured proceeding overall and at least one party outcome associated with a party to a proceeding corresponding to the one or more docket entries, wherein the outcome detector configured to analyze the one or more docket entries includes the outcome detector further configured to apply, by a deep learning classifier of the outcome detector, a hierarchical attention network approach to the one or more docket entries, wherein the outcome detector configured to apply the hierarchical attention network approach includes the outcome detector further configured to:
provide each word token of the one or more docket entries to a first word-level recurrent neural network (RNN),
provide an output of each corresponding first word-level RNN to a word-level attention layer configured to encode the one or more docket entries,
provide the encoded one or more docket entries to a second word-level RNN, and
provide an output of the second word-level RNN to a docket entry-level attention layer configured to encode a docket; and

an outcome prediction detector configured to generate, by, one or more docket entry outcome predictions corresponding to the one or more docket entries based on the encoded docket and the output of the second word-level RNN;
and
a final outcome detector configured to generate a final outcome based on the final outcome location prediction and the one or more docket entry outcome predictions.

10. The system of claim 9 , wherein the outcome detector includes a docket entry classifier, and wherein the docket entry classifier is configured to:
analyze the one or more docket entries individually;
determine whether individual docket entries include an individual outcome associated with a respective individual docket entry; and
identify the individual outcome associated with a respective individual docket entry, wherein identifying the individual outcome associated with a respective individual docket entry is performed without considering other docket entries of the at least one docket entry.

11. The system of claim 9 , wherein the deep learning classifier is further configured to:
analyze the one or more docket entries;
determine whether individual docket entries of the o

Assignees

• Thomson Reuters Entpr Centre Gmbh

Inventors

• Vacek Thomas
• Song Dezhao
• Nugent Tim
• Cowling Conner
• Teo Ronald
• Schilder Frank

Classifications

• G06N3/08

  Learning methods · CPC title

• G06Q50/18 Primary

  Legal services · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title