Domain specific representation of document text for accelerated natural language processing

Patent metadata

Field	Value
Publication number	US-9898447-B2
Application number	US-201514746327-A
Country	US
Kind code	B2
Filing date	Jun 22, 2015
Priority date	Jun 22, 2015
Publication date	Feb 20, 2018
Grant date	Feb 20, 2018

Abstract

Provided are techniques for a domain specific representation of document text for accelerated natural language processing. A document is selected from a set of documents to be analyzed. A character stream from the document is converted into a token stream based on tokenization rules. Irrelevant tokens are removed from the token stream. The tokens remaining in the token stream are converted into an integer domain representation based on a domain specific ontology dictionary. The integer domain representation are stored to a Graphics Processing Unit (GPU) processing queue of each of one or more GPUs. Then, a result set is received from the one or more GPUs.

First claim

What is claimed is:

1. A computer program product, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code executable by one of a Central Processing Unit (CPU) processor and at least one Graphics Processing Unit (GPU) processor to perform:
selecting a document from a set of documents to be analyzed;
converting a character stream from the document into a token stream based on tokenization rules;
removing irrelevant tokens from the token stream;
converting tokens remaining in the token stream into an integer domain representation based on a domain specific ontology dictionary, wherein each of the tokens is mapped to an integer based on mappings in the domain specific ontology dictionary;
storing the integer domain representation to a Graphics Processing Unit (GPU) processing queue of each of one or more GPUs;
receiving a result set from the one or more GPUs, wherein the result set includes tuples of 1) a specific pattern and 2) an offset from a beginning of the integer domain representation, and wherein the result set is generated using a compiled super Regular Expression (REGEX) that is compiled using the domain specific ontology dictionary;
storing the result set into an index for use in processing a search query; and
persisting the integer domain representation once for the document for processing of the document with a different compiled super REGEX.

2. The computer program product of claim 1 , wherein the program code is executable by at least one processor to perform:
persisting to the domain specific ontology dictionary any newly discovered relevant integer domain representations for new tokens for the document that are relevant to a domain of the domain specific ontology dictionary.

3. The computer program product of claim 1 , wherein a Software as a Service (SaaS) is configured to perform computer program product operations.

4. The computer program product of claim 1 , wherein the program code is executable by at least one processor to perform:
selecting a REGEX rule from a set of character based domain REGEX rules;
applying the domain specific ontology dictionary to map a character based REGEX to an integer based REGEX;
compiling the integer based REGEX; and
adding the compiled integer based REGEX to the super REGEX.

5. The computer program product of claim 1 , wherein the domain specific ontology dictionary is selected from a set of domain specific ontology dictionaries for different domains.

6. The computer program product of claim 1 , wherein the domain specific ontology dictionary represents domain terminology as a fixed set of integers.

7. The computer program product of claim 1 , wherein the program code is executable by at least one processor to perform:
assigning a group of tokens to a group, which is assigned one integer domain representation.

8. The computer program product of claim 1 , wherein the program code is executable by at least one processor to perform:
parallelizing document level extraction using REGEX processing with GPU acceleration over integers in the integer domain representation.

9. A computer system, comprising:
one or more Central Processing Unit (CPU) processors and Graphics Processing Unit (GPU) processors, one or more computer-readable memories and one or more computer-readable, tangible storage devices; and
program instructions, stored on at least one of the one or more computer-readable, tangible storage devices for execution by at least one of the one or more CPU processors via at least one of the one or more memories, to perform operations comprising:
selecting a document from a set of documents to be analyzed;
converting a character stream from the document into a token stream based on tokenization rules;
removing irrelevant tokens from the token stream;
converting tokens remaining in the token stream into an integer domain representation based on a domain specific ontology dictionary, wherein each of the tokens is mapped to an integer based on mappings in the domain specific ontology dictionary;
storing the integer domain representation to a Graphics Processing Unit (GPU) processing queue of each of one or more GPUs;
receiving a result set from the one or more GPUs, wherein the result set includes tuples of 1) a specific pattern and 2) an offset from a beginning of the integer domain representation, and wherein the result set is generated using a compiled super Regular Expression (REGEX) that is compiled using the domain specific ontology dictionary;
storing the result set into an index for use in processing a search query; and
persisting the integer domain representation once for the document for processing of the document with a different compiled super REGEX.

10. The computer system of claim 9 , wherein the operations further comprise:
persisting to the domain specific ontology dictionary any newly discovered relevant integer domain representations for new tokens for the document that are relevant to a domain of the domain specific ontology dictionary.

11. The computer system of claim 9 , wherein a Software as a Service (SaaS) is configured to perform computer system operations.

12. The computer system of claim 9 , wherein the operations further comprise:
selecting a REGEX rule from a set of character based domain REGEX rules;
applying the domain specific ontology dictionary to map a character based REGEX to an integer based REGEX;
compiling the integer based REGEX; and
adding the compiled integer based REGEX to the super REGEX.

13. The computer system of claim 9 , wherein the domain specific ontology dictionary is selected from a set of domain specific ontology dictionaries for different domains.

14. The computer system of claim 9 , wherein the domain specific ontology dictionary represents domain terminology as a fixed set of integers.

15. The computer system of claim 9 , wherein the operations further comprise:
assigning a group of tokens to a group, which is assigned one integer domain representation.

16. The computer system of claim 9 , wherein the operations further comprise:
parallelizing document level extraction using REGEX processing with GPU acceleration over integers in the integer domain representation.

Assignees

• IBM

Inventors

• Desai Rajesh M
• Housfater Alon S
• Parker Philip E
• Raphael Roger C

Classifications

• G06F40/284

  Lexical analysis, e.g. tokenisation or collocates · CPC title

• G06F16/367

  Ontology · CPC title

• G06F40/151 Primary

  Transformation · CPC title

• G06F16/2228

  Indexing structures · CPC title

• G06F9/46

  Multiprogramming arrangements · CPC title