Streaming parser for structured data-interchange files

Patent metadata

Field	Value
Publication number	US-2020089750-A1
Application number	US-201816132717-A
Country	US
Kind code	A1
Filing date	Sep 17, 2018
Priority date	Sep 17, 2018
Publication date	Mar 19, 2020
Grant date	—

Abstract

An embodiment may involve a computing system that includes a processor and memory. The memory may contain program instructions executable by the processor to repeatedly perform, for each block of a textual data-interchange file, operations including: obtaining a block of the file, where the block contains one or more records each containing one or more elements; identifying any pre-defined elements contained in records that are completed within the block, where the pre-defined elements are specified by a set of paths, the paths each hierarchically defining a location of an element within a record; storing, and into one or more files or one or more database tables, the pre-defined elements contained in records that are completed within the block; and determining whether the block ends with a partial record, and maintaining any such partial record for later storage in conjunction with processing of a subsequent block of the file.

First claim

What is claimed is:

1 . A computing system comprising:
a processor; and memory containing program instructions executable by the processor to repeatedly perform, for each block of a textual data-interchange file, operations including:
obtaining, by a parser executing on the computing system, a block of the textual data-interchange file, wherein the block contains one or more records and the one or more records each contain one or more elements;
identifying, by the parser, any pre-defined elements contained in records that are completed within the block, wherein the pre-defined elements are specified by a set of paths, the paths each hierarchically defining a location of an element within a record;
storing, by the parser and into one or more files or one or more database tables, the pre-defined elements contained in records that are completed within the block; and
determining, by the parser, whether the block ends with a partial record, and maintaining any such partial record for later storage in conjunction with processing of a subsequent block of the textual data-interchange file.

2 . The computing system of claim 1 , wherein the subsequent block follows the block in the textual data-interchange file.

3 . The computing system of claim 2 , wherein the subsequent block immediately follows the block in the textual data-interchange file.

4 . The computing system of claim 1 , wherein the textual data-interchange file is a JavaScript Object Notation (JSON) file.

5 . The computing system of claim 1 , wherein the pre-defined elements include at least one of objects, arrays, or values, wherein objects comprise name/value pairs, and wherein arrays comprise lists of values.

6 . The computing system of claim 1 , wherein the computing system receives the subsequent block while the parser is processing the block.

7 . The computing system of claim 1 , wherein the block contains a plurality of complete records and ends with the partial record, wherein storing the pre-defined elements contained in records that are completed within the block comprises:
storing the pre-defined elements in the plurality of complete records but not those of the partial record.

8 . The computing system of claim 1 , wherein the block begins with an additional partial record, contains a plurality of complete records, and ends with the partial record, and wherein storing the pre-defined elements contained in records that are completed within the block comprises:
retrieving a previously maintained partial record related to the additional partial record; combining the previously maintained partial record and the additional partial record to create an additional complete record; and storing the pre-defined elements in the additional complete record and the plurality of complete records.

9 . The computing system of claim 1 , wherein during the repeated performance of the operations, the parser uses no more than two blocks and one record worth of the memory for temporary storage of information from the textual data-interchange file.

10 . The computing system of claim 1 , wherein the pre-defined elements do not include all elements within the textual data-interchange file.

11 . The computing system of claim 1 , wherein the computing system is within a computational instance of a remote network management platform that uses the textual data-interchange file for communication with other devices.

12 . A computer-implemented method comprising repeatedly performing, for each block of a textual data-interchange file, operations including:
obtaining, by a parser executing on a computing system, a block of the textual data-interchange file, wherein the block contains one or more records and the one or more records each contain one or more elements; identifying, by the parser, any pre-defined elements contained in records that are completed within the block, wherein the pre-defined elements are specified by a set of paths, the paths each hierarchically defining a location of an element within a record; storing, by the parser and into one or more files or one or more database tables, the pre-defined elements contained in records that are completed within the block; and determining, by the parser, that the block ends with a partial record, and maintaining the partial record for later storage in conjunction with processing of a subsequent block of the textual data-interchange file.

13 . The computer-implemented method of claim 12 , wherein the computing system receives the subsequent block while the parser is processing the block.

14 . The computer-implemented method of claim 12 , wherein the block contains a plurality of complete records and ends with the partial record, wherein storing the pre-defined elements contained in records that are completed within the block comprises:
storing the pre-defined elements in the plurality of complete records but not those of the partial record.

15 . The computer-implemented method of claim 12 , wherein the block begins with an additional partial record, contains a plurality of complete records, and ends with the partial record, and wherein storing the pre-defined elements contained in records that are completed within the block comprises:
retrieving a previously maintained partial record related to the additional partial record; combining the previously maintained partial record and the additional partial record to create an additional complete record; and storing the pre-defined elements in the additional complete record and the plurality of complete records.

16 . The computer-implemented method of claim 12 , wherein during the repeated performance of the operations, the parser uses no more than two blocks and one record worth of memory for temporary storage of information from the textual data-interchange file.

17 . An article of manufacture including a non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by a computing system, cause the computing system to perform operations comprising:
obtaining, by a parser, a block of a textual data-interchange file, wherein the block contains one or more records and the one or more records each contain one or more elements; identifying, by the parser, any pre-defined elements contained in records that are completed within the block, wherein the pre-defined elements are specified by a set of paths, the paths each hierarchically defining a location of an element within a record; storing, by the parser and into one or more files or one or more database tables, the pre-defined elements contained in records that are completed within the block; and determining, by the parser, whether the block ends with a partial record, and maintaining any such partial record for later storage in conjunction with processing of a subsequent block of the textual data-interchange file.

18 . The article of manufacture of claim 17 , wherein the block contains a plurality of complete records and ends with a partial record, wherein storing the pre-defined elements contained in records that are completed within the block comprises:
storing the pre-defined elements in the plurality of complete records but not those of the partial record.

19 . The article of manufacture of claim 17 , wherein the block begins with an additional partial record, contains a plurality of complete records, and ends with the partial record, and wherein storing the pre-defined elements contained in records that are completed within the block comprises:
retr

Assignees

• Servicenow Inc

Inventors

• Ros Fernando
• Motamedi Khosrow Jian
• Krasnow Gregory Allen
• Bell Douglas Andrew

Classifications

• G06F16/24568

  Data stream processing; Continuous queries · CPC title

• G06F40/221 Primary

  Parsing markup language streams (streaming G06F40/149) · CPC title

• G06F40/131

  Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F40/149

  Adaptation of the text data for streaming purposes, e.g. Efficient XML Interchange [EXI] format · CPC title