Systems and methods for discovery, classification, and indexing of data in a native computing system

Patent metadata

Field	Value
Publication number	US-12259882-B2
Application number	US-202318312498-A
Country	US
Kind code	B2
Filing date	May 4, 2023
Priority date	Jan 25, 2021
Publication date	Mar 25, 2025
Grant date	Mar 25, 2025

Abstract

In general, various aspects provide methods, apparatuses, systems, computing devices, computing entities, and/or the like for performing data discovery on a target computing system. In various aspects, a third party computing connects, via a public data network, to an edge node of the target computing system and instructs the target computing system to execute jobs to discover target data stored in data repositories in a private data network in the target computing system. In some aspects, the third party computing system may schedule the jobs on the target computing system based on computing resource availability on the target computing system.

First claim

What is claimed is:

1. A target computing system comprising:
one or more computing devices having access to data at a plurality of data sources on a private data network; at least one processing device of an external-facing subsystem, wherein the external-facing subsystem facilitates a connection between the private data network and a third-party computing data discovery system; and
a non-transitory computer-readable medium comprising instructions that, when executed by the at least one processing device of the external-facing subsystem, cause the at least one processing device to perform operations comprising:
deploying executable code comprising a neural network based classification model, of the third-party computing data discovery system, to the one or more computing devices corresponding to the plurality of data sources on the private data network to perform scanning and classification operations at the plurality of data sources utilizing the neural network based classification model of the external-facing subsystem without accessing the data at the plurality of data sources on the private data network;
causing the one or more computing devices corresponding to the plurality of data sources to execute the executable code of the external-facing subsystem to scan the plurality of data sources for target data;
causing the one or more computing devices corresponding to the plurality of data sources to execute the executable code of the external-facing subsystem to utilize the neural network based classification model with the target data to generate data type predictions indicating data type labels for the target data by:
generating a prediction score of a particular data type for a data item of the target data based on tokenized data or labelled data corresponding to the target data; and
identifying the particular data type as a data type label for the data item of the target data based on the prediction score satisfying a confidence score threshold; and

responsive to scanning and classifying the target data stored on the plurality of data sources, generating and storing metadata for the plurality of data sources, the metadata indicating types of the target data.

2. The target computing system of claim 1 , wherein deploying the executable code to the one or more computing devices comprises:
executing a client application to expose online functions to a third-party system; and
deploying, utilizing the client application, the executable code to the one or more computing devices to perform the scanning and classification operations in connection with one or more operations at the third-party system.

3. The target computing system of claim 2 , wherein causing the one or more computing devices to execute the executable code to scan the plurality of data sources for the target data comprises:
generating a job schedule comprising a plurality of jobs based on computing resource data of the one or more computing devices; and
causing the one or more computing devices to execute the executable code to scan the plurality of data sources according to the plurality of jobs in the job schedule and the computing resource data of the one or more computing devices.

4. The target computing system of claim 3 , wherein causing the one or more computing devices to execute the executable code to scan the plurality of data sources for the target data comprises causing the one or more computing devices to execute one or more scripts associated with each job of the plurality of jobs of the job schedule.

5. The target computing system of claim 3 , wherein generating the job schedule comprises scheduling the plurality of jobs to limit a number of simultaneous jobs at the one or more computing devices to a particular number of simultaneous jobs according to the computing resource data.

6. The target computing system of claim 3 , wherein the operations further comprise determining the computing resource data by determining current resource usage by the one or more computing devices and total available resources of the one or more computing devices.

7. The target computing system of claim 1 , wherein causing the one or more computing devices to execute the executable code to generate the data type predictions for the target data comprises causing the one or more computing devices to generate the tokenized data by tokenizing the data of the plurality of data sources.

8. The target computing system of claim 7 , wherein causing the one or more computing devices to generate the data type predictions indicating the data type labels comprises causing the one or more computing devices to:
generate labelled data for the data item by labeling the tokenized data; and
identify the data type label for the data item by generating the data type predictions for the target data based on the labelled data.

9. The target computing system of claim 1 , wherein generating and storing the metadata for the plurality of data sources comprises:
generating metadata comprising a predicted data type label for the data item of the target data and a storage location of the data item of the target data at a data source of the plurality of data sources; and
storing the metadata comprising a predicted data type label for the data item of the target data and the storage location of the data item of the target data at the data source of the plurality of data sources.

10. A method comprising:
deploying, by an external-facing subsystem of a target computing system, executable code comprising a neural network based classification model, of a third-party computing data discovery system, to one or more computing devices corresponding to a plurality of data sources on a private data network of the target computing system having access to data at a plurality of data sources of the target computing system on the private data network to perform scanning and classification operations at the plurality of data sources utilizing the neural network based classification model of the external-facing subsystem without accessing the data at the plurality of data sources on the private data network, wherein the external-facing subsystem facilitates a connection between the private data network and the third-party computing data discovery system;
generating, by the external-facing subsystem, a job schedule comprising a plurality of jobs corresponding to a plurality of data items at the plurality of data sources;
causing, by the external-facing subsystem, the one or more computing devices of the target computing system corresponding to the plurality of data sources to execute the executable code of the external-facing subsystem to scan the plurality of data sources for target data according to the plurality of jobs of the job schedule;
causing, by the external-facing subsystem, the one or more computing devices of the target computing system corresponding to the plurality of data sources to execute the executable code of the external-facing subsystem to utilize the neural network based classification model with the target data to generate classifications of data types for the target data at the plurality of data sources by:
generating a prediction score of a particular data type for a data item of the target data based on tokenized data or labelled data corresponding to the target data; and
identifying a particular data type as a data type label for the data item of the target data based on the prediction score satisfying a confidence score threshold; and

responsive to scanning and classifying the target data stored on the plurality of data sources, generating and storing metadata for the plurality of data sources, the metadata indicating types of the target data and storage locati

Assignees

• Onetrust Llc

Inventors

• Raghupathy Haribalan
• Pitchaimani Saravanan
• Lynn Jonathan
• Shinde Rahul
• Jones Kevin
• Viswanathan Subramanian
• Sivan Mahesh
• Dana Zara
• Shah Milap
• Chandramohan Sivanandame
• UPADHYAY Abhishek
• Balasubramanian Anand

Classifications

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06Q10/063

  Operations research, analysis or management · CPC title

• G06F9/5055

  considering software capabilities, i.e. software resources associated or available to the machine · CPC title

• G06F40/284

  Lexical analysis, e.g. tokenisation or collocates · CPC title

• G06F16/22

  Indexing; Data structures therefor; Storage structures · CPC title