Order independent data categorization, indication, and remediation across realtime datasets of live service environments

Patent metadata

Field	Value
Publication number	US-11789982-B2
Application number	US-202017029524-A
Country	US
Kind code	B2
Filing date	Sep 23, 2020
Priority date	Sep 23, 2020
Publication date	Oct 17, 2023
Grant date	Oct 17, 2023

Abstract

A computer-implemented method is provided of finding one or more data items that match one or more defined criteria in a dataset. The method comprises identifying data snippets of the dataset using a set of one or more attention rules; categorizing the identified data snippets using fuzzy matching by assigning them to buckets such that each bucket contains data snippets that are similar to another according to a similarity measure; classifying buckets containing data snippets having more than a threshold number of the true positive data items as true positive buckets and remaining buckets as false positive buckets; calculating culling rules based on the true positive buckets and the false positive buckets, and using the culling rules to remove the false positive data items from the true positive buckets.

First claim

The invention claimed is:

1. An order independent computer-implemented method of auditing a large-scale dataset for sensitive data, the method comprising:
receiving content from a real-time dataset of a live service environment;
identifying data snippets of the real-time dataset using a set of one or more attention rules, wherein the one or more attention rules are opportunistic attention rules configured for maximising a recall rate, and each identified data snippet includes either a true positive data item or a false positive data item, wherein a true positive data item is a data item that matches one or more defined criteria associated with sensitive data, and a false positive data item is a data item that does not match the one or more defined criteria associated with sensitive data, wherein the recall rate indicates a portion of identified true positive data items that match the one or more defined criteria associated with sensitive data among true positive data items present in the received content;
categorizing the identified data snippets using fuzzy hashing by assigning them to buckets such that each bucket contains data snippets that are similar to one another according to a similarity measure defined by a fuzzy hashing algorithm for performing the fuzzy hashing;
classifying buckets containing data snippets having more than a threshold number of the true positive data items as true positive buckets and remaining buckets as false positive buckets;
calculating culling rules based on the true positive buckets and the false positive buckets, wherein the culling rules are configured to reduce a total number of false positive data items in the true positive buckets to increase a precision rate; and
using the culling rules to remove the false positive data items that do not meet the one or more defined criteria associated with sensitive data from the true positive buckets, to increase the precision rate that corresponds to a portion of true positive data items that meet the one or more defined criteria associated with sensitive data among data items in the true positive buckets.

2. The computer-implemented method of claim 1 , wherein the one or more defined criteria comprises sensitivity of information.

3. The computer-implemented method of claim 1 , comprising receiving the real-time dataset in a form of a stream.

4. The computer-implemented method of claim 1 , wherein the fuzzy hashing is performed using SSDeep, SDHash, Nilsimsa or TLSH.

5. The computer-implemented method of claim 4 , wherein two data snippets are considered to be similar and assigned to the same bucket if a comparison score above a threshold has been calculated for the two data snippets.

6. The computer-implemented method of claim 1 , wherein the set of one or more attention rules are adapted over time.

7. A non-transitory computer-readable medium comprising computer executable instructions stored thereon which, when executed by one or more processors cause the processors to: perform an auditing method of finding one or more data items that match one or more defined criteria associated with sensitive data in a large-scale dataset, the method comprising:
identifying data snippets of the dataset using a set of one or more attention rules, wherein the one or more attention rules are opportunistic attention rules configured for maximising a recall rate, and each identified data snippet includes either a true positive data item or a false positive data item, wherein a true positive data item is a data item that matches the one or more defined criteria associated with sensitive data, and a false positive data item is a data item that does not match the one or more defined criteria associated with sensitive data, wherein the recall rate indicates a portion of identified true positive data items that match the one or more defined criteria associated with sensitive data among true positive data items present in content of the dataset;
categorizing the identified data snippets using fuzzy hashing by assigning them to buckets such that each bucket contains the data snippets that are similar to another according to a similarity measure defined by a fuzzy hashing algorithm for performing the fuzzy hashing;
classifying the buckets containing data snippets having more than a threshold number of the true positive data items as true positive buckets and remaining buckets as false positive buckets;
calculating culling rules based on the true positive buckets and the false positive buckets, wherein the culling rules are configured to reduce a total number of false positive data items in the true positive buckets to increase a precision rate; and
using the culling rules to remove the false positive data items that do not meet the one or more defined criteria associated with sensitive data from the true positive buckets, to increase the precision rate that corresponds to a portion of true positive data items that meet the one or more defined criteria associated with sensitive data among data items in the true positive buckets.

8. The non-transitory computer-readable medium of claim 7 , wherein the culling rules are calculated in such a way that the recall rate is maximised while also keeping the precision rate as high as possible.

9. The non-transitory computer-readable medium of claim 7 , wherein the culling rules are calculated such that a minimum set of culling rules maximises an amount of culling done on the true positive buckets.

10. The non-transitory computer-readable medium of claim 7 , wherein the culling rules are calculated by calculating a subset of features that are present in all data snippets of the false positive buckets but are not present in the true positive buckets.

11. The non-transitory computer-readable medium of claim 7 , wherein the culling rules are considered to be precise.

12. The non-transitory computer-readable medium of claim 7 , wherein the culling rules are recalculated at a point of time that is determined using a Fast Fourier Transformation.

13. The non-transitory computer-readable medium of claim 7 , wherein the culling rules are recalculated at a point of time that is determined by ratio of new data added to all data known.

14. A computer system, comprising
one or more processors;
one or more non-transitory computer-readable media having computer-executable instructions stored on the one or more processors, wherein the one or more computer-executable instructions are configured to cause the one or more processors to perform an auditing method of finding one or more data items that match one or more defined criteria associated with sensitive data in a large-scale dataset, the method comprising:
identifying data snippets of the dataset using a set of one or more attention rules, wherein the one or more attention rules are opportunistic attention rules configured for maximising a recall rate, and each identified data snippet includes either a true positive data item or a false positive data item, wherein a true positive data item is a data item that matches one or more defined criteria associated with sensitive data, and a false positive data item is a data item that does not match the one or more defined criteria associated with sensitive data, wherein the recall rate indicates a portion of identified true positive data items that match the one or more defined criteria associated with sensitive data among true positive data items present in content of the dataset;
categorizing the identified data snippets using fuzzy hashing by assigning them to buckets such that each bucket contains data snippets that are similar to another according to a similarity me

Assignees

• Electronic Arts Inc

Inventors

• Tumpic Denis
• Schafer Brian
• Nix James
• Aofolaju Shina
• Campbell Jesse

Classifications

• G06F16/285 Primary

  Clustering or classification · CPC title

• G06F16/2255

  Hash tables · CPC title

• G06F16/2468 Primary

  Fuzzy queries · CPC title

• G06N7/023

  Learning or tuning the parameters of a fuzzy system · CPC title

• G06N5/048

  Fuzzy inferencing · CPC title