Method and system for fuzzy matching and alias matching for streaming data sets

Patent metadata

Field	Value
Publication number	US-11704387-B2
Application number	US-202017006384-A
Country	US
Kind code	B2
Filing date	Aug 28, 2020
Priority date	Aug 28, 2020
Publication date	Jul 18, 2023
Grant date	Jul 18, 2023

Abstract

A method, system, and computer-usable medium for streaming or processing data streams. Raw text data is cleansed to a standard format. A fuzzy matching algorithm is performed on the text data. For data where domain expertise is required, alias matching is performed. End state categorizing or grouping is provided for the cleansed raw text data.

First claim

What is claimed is:

1. A computer-implementable method for streaming data streams comprising:
cleansing raw text data of the data streams;
applying a fuzzy matching algorithm on input strings of cleansed raw text data to a list of viable end state category or group matches based on common sense level comparisons approximating matching ability and confidence levels of individuals with no domain expertise in a field in which the text data applies, to determine an end state category of the cleansed raw text data;
performing alias matching on the cleansed raw text data that has been applied with the fuzzy matching algorithm, if domain expertise is determined to be used for cleansed raw text data;
separating fuzzy and alias matching into different pipeline stages along expertise boundaries; and
providing the end state category of the cleansed raw text data based on the applying the fuzzy matching algorithm.

2. The method of claim 1 , wherein a configurable software template of a data management platform determines processes that are implemented for the streaming data streams.

3. The method of claim 1 , wherein the raw text data is received from different data sources.

4. The method of claim 1 , wherein the fuzzy matching algorithm is Levenshtein edit distance.

5. The method of claim 1 , wherein the fuzzy matching algorithm implements a confidence level threshold.

6. The method of claim 1 , wherein the alias matching implements a lookup table.

7. The method of claim 1 , wherein a best match is performed for the fuzzy matching and alias matching.

8. A system comprising:
a processor;
a data bus coupled to the processor; and
a non-transitory, computer-readable storage medium embodying computer program code, the non-transitory, computer-readable storage medium being coupled to the data bus, the computer program code interacting with a plurality of computer operations and comprising instructions executable by the processor and configured for:
cleansing raw text data of the data streams;
applying a fuzzy matching algorithm on input strings of cleansed raw text data to a list of viable end state category or group matches based on common sense level comparisons approximating matching ability and confidence levels of individuals with no domain expertise in a field in which the text data applies, to determine an end state category of the cleansed raw text data;
performing alias matching on the cleansed raw text data that has been applied with the fuzzy matching algorithm, if domain expertise is determined to be used for cleansed raw text data;
separating fuzzy and alias matching into different pipeline stages along expertise boundaries; and
providing the end state category of the cleansed raw text data based on the applying the fuzzy matching algorithm.

9. The system of claim 8 , wherein a configurable software template of a data management platform determines processes that are implemented for the streaming data streams.

10. The system of claim 8 , wherein the raw text data is received from different data sources.

11. The system of claim 8 , wherein the fuzzy matching algorithm is Levenshtein edit distance.

12. The system of claim 8 , wherein the fuzzy matching algorithm implements a confidence level threshold.

13. The system of claim 8 , wherein the alias matching implements a lookup table.

14. The system of claim 8 , wherein a best match is performed for the fuzzy matching and alias matching.

15. A non-transitory, computer-readable storage medium embodying computer program code, the computer program code comprising computer executable instructions configured for:
cleansing raw text data of the data streams;
applying a fuzzy matching algorithm on input strings of cleansed raw text data to a list of viable end state category or group matches based on common sense level comparisons approximating matching ability and confidence levels of individuals with no domain expertise in a field in which the text data applies, to determine an end state category of the cleansed raw text data;
performing alias matching on the cleansed raw text data that has been applied with the fuzzy matching algorithm, if domain expertise is determined to be used for cleansed raw text data;
separating fuzzy and alias matching into different pipeline stages along expertise boundaries; and
providing the end state category of the cleansed raw text data based on the applying the fuzzy matching algorithm.

16. The non-transitory, computer-readable storage medium of claim 15 , wherein a configurable software template of a data management platform determines processes that are implemented for the streaming data streams.

17. The non-transitory, computer-readable storage medium of claim 15 , wherein the raw text data is received from different data sources.

18. The non-transitory, computer-readable storage medium of claim 15 , wherein the fuzzy matching algorithm is Levenshtein edit distance.

19. The non-transitory, computer-readable storage medium of claim 15 , wherein the fuzzy matching algorithm implements a confidence level threshold.

20. The non-transitory, computer-readable storage medium of claim 15 wherein the alias matching implements a lookup table.

Assignees

• Forcepoint Llc

Inventors

• Stewart Peter Lochlan
• Kinnischtzke Amanda

Classifications

• G06F18/22 Primary

  Matching criteria, e.g. proximity measures · CPC title

• G06F16/24568

  Data stream processing; Continuous queries · CPC title

• G06F40/10 Primary

  Text processing (natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40) · CPC title

• G06N5/048

  Fuzzy inferencing · CPC title

• G06N20/00

  Machine learning · CPC title