Machine learning model for predicting litigation risk in correspondence and identifying severity levels

Patent metadata

Field	Value
Publication number	US-11615361-B2
Application number	US-202117307163-A
Country	US
Kind code	B2
Filing date	May 4, 2021
Priority date	Sep 13, 2019
Publication date	Mar 28, 2023
Grant date	Mar 28, 2023

Abstract

Systems, methods, and other embodiments associated with detecting severity levels of risk in an electronic correspondence are described. In one embodiment, a method includes inputting, into a memory, a target electronic correspondence that has been classified as being litigious by a machine learning classifier. An artificial intelligence rule-based technique is applied to the target electronic correspondence that identifies high and medium risk level keywords. The technique is also configured to generate a litigious score based on a sum of term frequencies-inverse document frequencies using the remaining keywords. An electronic notice is transmitted to a remote computer over a communication network that identifies the target electronic correspondence and the level of litigation risk.

First claim

What is claimed is:

1. A computing system, comprising:
at least one processor configured to execute instructions;
at least one memory operably connected to the at least one processor and configured to at least store and provide computer-executable instructions to the at least one processor;
a machine learning classifier configured to identify terminology from text of electronic correspondence and to classify the electronic correspondence with a risk as being litigious or non-litigious based at least on the terminology;
a non-transitory computer-readable medium that includes stored thereon computer-executable instructions that when executed by the at least one processor cause the computing system to:
input, into the at least one memory, a target electronic correspondence that has been classified as being litigious by the machine learning classifier;
perform a first rule that searches the text of the target electronic correspondence for one or more first designated keywords that represent high litigious risk;
wherein the one or more first designated keywords are defined in a library of keywords stored in a data structure;
in response to the one or more first designated keywords being found in the text, mark the target electronic correspondence as a high level of litigation risk and generate a first output of the mark;
in response to the one or more first designated keywords not being found in the text:
(i) perform a second rule that searches the text of the target electronic correspondence for one or more second designated keywords that represent medium litigious risk and are defined in the library of keywords; and
(ii) in response to the one or more second designated keywords being found in the text, mark the target electronic correspondence as a medium level of litigation risk and generate a second output of the mark;

in response to the one or more second designated keywords not being found in the text:
(i) perform a third rule that searches the text of the target electronic correspondence for remaining keywords that have a match in the library of keywords;
(ii) generate a litigious score based on a sum of term frequencies-inverse document frequencies using the remaining keywords; and
(iii) based on the litigious score as compared to a threshold, mark the target electronic correspondence with a level of litigation risk; and

transmit an electronic notice to a remote computer over a communication network that identifies the target electronic correspondence and the level of litigation risk.

2. The computing system of claim 1 , wherein marking the target electronic correspondence with a level of litigation risk based on the litigious score further includes instructions that when executed by at least the processor cause the processor to:
mark the target electronic correspondence as a high level of litigation risk when the litigious score is above the threshold;
mark the target electronic correspondence as a medium level of litigation risk when the litigious score is between a range of values below the threshold; and
mark the target electronic correspondence as a low level of litigation risk when the litigious score is below the range of values.

3. The computing system of claim 1 , wherein the library of keywords comprises a list of keywords wherein each keyword is assigned as representing a high level, a medium level, or a low level of litigation risk.

4. The computing system of claim 1 , wherein the computer-executable instructions are further configured to:
generate an electronic notice to include identification of the target electronic correspondence, the level of ligation risk, and one or more of the keywords found in the text that match the first designated keywords or the second designated keywords;
generate a dashboard as part of a graphical user interface configured to display a list of electronic correspondences that have been identified and labeled with the level of litigation risk from received electronic notices; and
transmit the electronic notice to the dashboard of the graphical user interface that causes information from the electronic notice to be displayed on the dashboard.

5. The computing system of claim 1 , wherein the computer-executable instructions that generate the litigious score are further configured to:
remove stop words from the text of the target electronic correspondence; and
calculate the sum of term frequencies-inverse document frequencies using the remaining keywords.

6. The computing system of claim 1 , wherein the computer-executable instructions that transmit the electronic notice to a remote computer are further configured to:
transmit the electronic notice to the remote computer and cause a dashboard of a graphical user interface to display identification of the target electronic correspondence, the level of ligation risk, and one or more of the keywords found in the text.

7. The computing system of claim 1 , wherein the computer-executable instructions are further configured to:
generate a dashboard on a graphical user interface that includes a line graph representing a separate line that tracks a number of risks identified for each of the high level of litigation risk, the medium level of litigation risk, and the low level of litigation risk over a time period.

8. A non-transitory computer-readable medium that includes stored thereon computer-executable instructions that when executed by at least a processor of a computer cause the computer to:
input, into a memory, a target electronic correspondence that has been classified as being litigious by a machine learning classifier;
identify a first group of keywords from a library of keywords, wherein the first group of keywords indicate a high level of litigation risk;
identify a second group of keywords from the library of keywords, wherein the second group of keywords indicate a medium level of litigation risk;
access the memory and analyze text from the target electronic correspondence to determine a presence of terms that correspond to one or more of the first group of keywords or the second group of keywords;
when the presence of terms is found, generate an indication on a display screen that the target electronic correspondence is labeled as the high level of litigation risk or the medium level of litigation risk;
when the presence of terms is not found:
(i) search the text of the target electronic correspondence for remaining keywords that have a match in the library of keywords;
(ii) generate a litigious score based on a sum of term frequencies-inverse document frequencies using the remaining keywords; and
(iii) based on the litigious score as compared to a threshold, generate a label for the target electronic correspondence that indicates a level of litigation risk;

transmit an electronic notice to a remote computer over a communication network that identifies the target electronic correspondence and the level of litigation risk.

9. The non-transitory computer-readable medium of claim 8 , further comprising instructions that when executed by at least the processor cause the processor to:
execute the machine learning classifier configured to identify construction terminology and to classify text with a risk as being litigious or non-litigious.

10. The non-transitory computer-readable medium of claim 8 , wherein the instructions to analyze the text further comprise instructions that when executed by at least the processor cause the processor to:
perform a first rule that searches the text of the target electronic correspondence for one or more of the first group of keywords that represent the high level of litigious risk;
in response to the one or more of the first group of

Assignees

• Oracle Int Corp

Inventors

• Nag Ria
• Nambiar Padmakumar
• Venkatasubramanian Karthik
• Praharaj Suvendu

Classifications

• G06F40/30

  Semantic analysis · CPC title

• G06Q10/10

  Office automation; Time management · CPC title

• G06F18/259

  Fusion by voting · CPC title

• G06Q10/107

  Computer-aided management of electronic mailing [e-mailing] · CPC title

• G06N20/20 Primary

  Ensemble learning · CPC title