Parallel parsing of markup language data

Patent metadata

Field	Value
Publication number	US-2016292147-A1
Application number	US-201615051698-A
Country	US
Kind code	A1
Filing date	Feb 24, 2016
Priority date	Mar 30, 2015
Publication date	Oct 6, 2016
Grant date	—

Abstract

Parsing XML (extensible markup language) data by performing the following operations: (i) dividing the piece of markup language into a plurality of pre-parsing segments; (ii) assigning the pre-parsing segment to a pre-parsing processor thread of a plurality of pre-parsing processor threads; (iii) determining any parsing division point(s) occurring in the pre-parsing segment so that data corresponding to a single tabular record is between each consecutive pair of parsing division points; (iv) dividing the piece of language into a plurality of parsing segments defined by the parsing division points so that each parsing segment corresponds to a single tabular record; (v) assigning the parsing segment to a parsing processor thread of a plurality of parsing processor threads; and (vi) parsing to generate a parsed tabular record corresponding to the parsing segment.

First claim

What is claimed is:

1 . A method of processing a piece of markup language by machine logic, the method comprising:
dividing the piece of markup language into a plurality of pre-parsing segments; for each pre-parsing segment, assigning the pre-parsing segment to a pre-parsing processor thread of a plurality of pre-parsing processor threads; for each pre-parsing segment, determining, by its assigned pre-parsing processor thread, any parsing division point(s) occurring in the pre-parsing segment so that data corresponding to a single tabular record is between each consecutive pair of parsing division points; responsive to the determination of the parsing division points, dividing the piece of language into a plurality of parsing segments defined by the parsing division points so that each parsing segment corresponds to a single tabular record; for each parsing segment, assigning the parsing segment to a parsing processor thread of a plurality of parsing processor threads; and for each parsing segment, parsing, by its assigned parsing processor thread, to generate a parsed tabular record corresponding to the parsing segment.

2 . The method of claim 1 wherein at least a substantial number of the pre-parsing segments have an equal number of lines.

3 . The method of claim 1 wherein at least a substantial number of the pre-parsing segments have an equal number of binary bits.

4 . The method of claim 1 wherein the division of the piece of markup language into a plurality of pre-parsing segments is performed by machine logic in the form of an analyze schema module.

5 . The method of claim 1 wherein the pre-parsing processor threads determine parsing division points using machine logic in the form of readers.

6 . The method of claim 1 wherein the markup language is XML.

Assignees

• IBM

Inventors

• Li Zhen
• Shum Mi Wan
• Wei Dongjie
• Wong Samuel H
• Wu Xian Rong

Classifications

• G06F16/3334 Primary

  Selection or weighting of terms from queries, including natural language queries · CPC title

• G06F40/221

  Parsing markup language streams (streaming G06F40/149) · CPC title

• G06F17/272 Primary

  Physics · mapped topic

• G06F17/2247

  Physics · mapped topic

• G06F40/143

  Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD] · CPC title