Authenticating data associated with a data intake and query system using a distributed ledger system

Patent metadata

Field	Value
Publication number	US-11921873-B1
Application number	US-202117305646-A
Country	US
Kind code	B1
Filing date	Jul 12, 2021
Priority date	Sep 26, 2018
Publication date	Mar 5, 2024
Grant date	Mar 5, 2024

Abstract

Systems and methods are disclosed for authenticating a chunk of data identified in a query received by a data intake and query system. The data intake and query system receives a query that identifies a set of data and manner for processing the set of data, and identifies a chunk of data that is part of the set of data. The system generates a content identifier, such as a hash, of the chunk of data. The system further authenticates the chunk of data based on the generated content identifier and a content identifier stored by a distributed ledger system.

First claim

What is claimed:

1. A method, comprising:
receiving, at a data intake and query system, a query that identifies a set of data and a manner of processing the set of data;
identifying a bucket containing data that satisfies a portion of the query, wherein a bucket is a data storage structure for storing files on a digital storage medium;
identifying a particular file of the bucket for use in executing the query; generating a first content identifier for the particular file based on content of the particular file; and
authenticating the particular file for the query based on a comparison of the first content identifier with a second content identifier for the particular file, wherein authenticating the particular file comprises:
communicating a file identifier of the particular file to a distributed ledger system, wherein the distributed ledger system uses the file identifier of the particular file to identify a block of a blockchain of the distributed ledger system that stores the second content identifier, and
receiving the second content identifier from the distributed ledger system.

2. The method of claim 1 , further comprising:
identifying one or more query parameters based on the query;
communicating the one or more query parameters to the distributed ledger system; and
receiving a plurality of content identifiers from the distributed ledger system including the second content identifier, wherein the distributed ledger system uses the one or more query parameters to identify block entries that satisfy the one or more query parameters.

3. The method of claim 1 , wherein the first content identifier comprises a hash of the particular file.

4. The method of claim 1 , wherein the first content identifier comprises a hash of a group of related files including the particular file.

5. The method of claim 1 , wherein the first content identifier comprises a hash of a plurality of files of the bucket.

6. The method of claim 1 , wherein the first content identifier comprises a hash of a plurality of hashes of a plurality of files of the bucket.

7. The method of claim 1 , further comprising obtaining the particular file from an external data system.

8. The method of claim 7 , wherein the second content identifier corresponds to a content identifier generated by the data intake and query system prior to the first content identifier.

9. The method of claim 1 , wherein the distributed ledger system stores the second content identifier with other content identifiers associated with buckets of data associated with a same field-value pair.

10. The method of claim 1 , wherein at least one block of the blockchain comprises a plurality of block entries including at least one block entry corresponding to the particular file.

11. The method of claim 1 , wherein at least one block of the blockchain comprises a plurality of block entries, each block entry corresponding to a bucket of data managed by the data intake and query system.

12. The method of claim 1 , wherein at least one block of the blockchain comprises a plurality of block entries, at least one block entry of the plurality of block entries comprising a plurality of content identifiers.

13. The method of claim 1 , wherein at least one block of the blockchain comprises a plurality of block entries associated with a same field-value pair.

14. The method of claim 1 , wherein the distributed ledger system stores a plurality of blocks of a blockchain, wherein each block of the blockchain comprises a plurality of content identifiers.

15. The method of claim 1 , wherein the distributed ledger system stores a plurality of blocks of a blockchain, wherein each block of the blockchain comprises a plurality of content identifiers and a time range associated with chunks of data associated with the plurality of content identifiers.

16. The method of claim 1 , wherein the distributed ledger system stores a plurality of blockchains, each blockchain comprising a plurality of blocks that store one or more content identifiers, wherein the plurality of blocks of a particular blockchain are associated with a same field-value pair, wherein the plurality of blockchains includes the blockchain.

17. A computing system of a data intake and query system, the computing system comprising:
memory; and
one or more processing devices coupled to the memory and configured to:
receive a query that identifies a set of data and a manner of processing the set of data;
identify a plurality of buckets containing data that satisfies a portion of the query, wherein a bucket is a data storage structure for storing files on a digital storage medium;
identify bucket containing data that satisfies the portion of the query, wherein a bucket is a data storage structure for storing files on a digital storage medium;
identify a particular file of the bucket for use in executing the query;
generate a first content identifier for the particular file based on content of the particular file; and
authenticate the particular file for the query based on a comparison of the first content identifier with a second content identifier for the particular file, wherein to authenticate the particular file, the one or more processing devices are configured to:
communicate a file identifier of the particular file to a distributed ledger system, wherein the distributed ledger system uses the file identifier of the particular file to identify a block of a blockchain of the distributed ledger system that stores the second content identifier, and
receive the second content identifier from the distributed ledger system.

18. Non-transitory computer readable media comprising computer-executable instructions that, when executed by a computing system of a data intake and query system, cause the computing system to:
receive a query that identifies a set of data and a manner of processing the set of data;
identify a plurality of buckets containing data that satisfies a portion of the query, wherein a bucket is a data storage structure for storing files on a digital storage medium;
identify bucket containing data that satisfies the portion of the query, wherein a bucket is a data storage structure for storing files on a digital storage medium;
identify a particular file of the bucket for use in executing the query;
generate a first content identifier for the particular file based on content of the particular file; and
authenticate the particular file for the query based on a comparison of the first content identifier with a second content identifier for the particular file, wherein to authenticate the particular file, the computer-executable instructions cause the computing system to:
communicate a file identifier of the particular file to a distributed ledger system, wherein the distributed ledger system uses the file identifier of the particular file to identify a block of a blockchain of the distributed ledger system that stores the second content identifier, and
receive the second content identifier from the distributed ledger system.

Assignees

• Splunk Inc

Inventors

• Mckervey Nathaniel Gerard
• Moore Ryan Russell

Classifications

• G06F21/6218 Primary

  to a system of files or objects, e.g. local or distributed file system or database · CPC title

• G06F16/137

  Hash-based (content-based indexing of textual data G06F16/31) · CPC title

• G06F16/245

  Query processing · CPC title

• H04L9/0637

  Modes of operation, e.g. cipher block chaining [CBC], electronic codebook [ECB] or Galois/counter mode [GCM] · CPC title

• H04L9/3239 Primary

  involving non-keyed hash functions, e.g. modification detection codes [MDCs], MD5, SHA or RIPEMD · CPC title