Mapping search nodes to a search head using a tenant identifier

Patent metadata

Field	Value
Publication number	US-11275733-B1
Application number	US-201916513546-A
Country	US
Kind code	B1
Filing date	Jul 16, 2019
Priority date	Apr 30, 2018
Publication date	Mar 15, 2022
Grant date	Mar 15, 2022

Abstract

Systems and methods are disclosed for mapping search nodes to a search head in a data intake and query system based on a tenant identifier in order to execute a query received by the data intake and query system. The mapping may allow same or similar search nodes to be used to execute queries that are associated with a particular tenant identifier, in order to take advantage of caching and local data stored with those search nodes. In some cases, search nodes can be mapped based on the tenant identifier using a hashing algorithm, such as a consistent hashing algorithm.

First claim

What is claimed is:

1. A method comprising:
receiving, by a data intake and query system, a query associated with a tenant identifier, the query identifying a set of data to be processed and a manner of processing the set of data;
identifying, by the data intake and query system, a search head to process the query;
dynamically mapping a first set of one or more search nodes from a plurality of search nodes to the search head based on the tenant identifier associated with the query wherein dynamically mapping the first set of one or more search nodes to the search head is based on receiving the query;
processing the query using the identified search head, wherein processing the query includes generating instructions for the first set of one or more search nodes dynamically mapped to the search head; and
executing the query, wherein executing the query includes communicating the generated instructions to the first set of one or more search nodes dynamically mapped to the search head, wherein the first set of one or more search nodes execute the instructions.

2. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:
receiving, by the data intake and query system, a second query associated with the tenant identifier, the second query identifying a second set of data to be processed and a manner of processing the second set of data;
identifying, by the data intake and query system, a different search head to process the second query;
dynamically mapping a second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein the second set of one or more search nodes includes at least one search node from the first set of one or more search nodes, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;
processing the query using the different search head; and
executing the query using the second set of one or more search nodes dynamically mapped to the different search head.

3. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:
receiving, by the data intake and query system, a second query associated with the tenant identifier, the second query identifying a second set of data to be processed and a manner of processing the second set of data;
identifying, by the data intake and query system, a different search head to process the second query;
dynamically mapping a second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein the second set of one or more search nodes has the same search nodes as the first set of one or more search nodes, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;
processing the query using the different search head; and
executing the query using the second set of one or more search nodes dynamically mapped to the different search head.

4. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:
receiving, by the data intake and query system, a second query associated with the tenant identifier, the second query identifying a second set of data to be processed and a manner of processing the second set of data;
identifying, by the data intake and query system, a different search head to process the second query;
dynamically mapping a second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;
processing the query using the different search head; and
executing the query using the second set of one or more search nodes dynamically mapped to the different search head,
wherein dynamically mapping the first set of one or more search nodes to the search head includes hashing the tenant identifier associated with the query using a hashing algorithm, and wherein dynamically mapping the second set of one or more search nodes to the different search head includes hashing the tenant identifier associated with the second query using the hashing algorithm.

5. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:
receiving, by the data intake and query system, a second query associated with the tenant identifier, the second query identifying a second set of data to be processed and a manner of processing the second set of data;
identifying, by the data intake and query system, a different search head to process the second query;
dynamically mapping a second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;
processing the query using the different search head; and
executing the query using the second set of one or more search nodes dynamically mapped to the different search head,
wherein dynamically mapping the first set of one or more search nodes to the search head includes hashing the tenant identifier associated with the query using a consistent hashing algorithm, and wherein dynamically mapping the second set of one or more search nodes to the different search head includes hashing the tenant identifier associated with the second query using the consistent hashing algorithm.

6. The method of claim 1 , further comprising:
mapping a first set of buckets to the first set of one or more search nodes based on a respective node identifier associated with each search node in the first set of one or more search nodes, wherein the first set of buckets includes the set of data to be processed,
wherein the first set of one or more search nodes execute the query using the first set of buckets mapped to the first set of one or more search nodes.

7. The method of claim 1 , wherein the query is a first query and the set of data is a first set of data, the method further comprising:
mapping a first set of buckets to the first set of one or more search nodes based on a respective node identifier associated with each search node in the first set of one or more search nodes, wherein the first set of buckets includes the first set of data to be processed, wherein the first set of one or more search nodes execute the query using the first set of buckets mapped to the first set of one or more search nodes;
receiving, by the data intake and query system, a second query associated with the tenant identifier, the second query identifying a second set of data to be processed and a manner of processing the second set of data;
identifying, by the data intake and query system, a different search head to process the second query;
dynamically mapping a second set of one or more search nodes from the plurality of search nodes to the different search head based on the tenant identifier associated with the second query, wherein dynamically mapping the second set of one or more search nodes to the different search head is based on receiving the second query;
processing the query using the different search head;
mapping a second set of buckets to the second set of one or more s

Assignees

• Splunk Inc

Inventors

• Batsakis Alexandros
• Calvert Scott
• James Alexander Douglas
• Li Bei
• Mathew Ashish
• Monschke James
• Moshtaghi Sogol
• Pride Christopher Madden
• Wang Xiaowei

Classifications

• G06F16/2455

  Query execution · CPC title

• G06F11/3433

  for load management (allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029) · CPC title

• G06F16/24539 Primary

  using cached or materialised query results · CPC title

• G06F16/137

  Hash-based (content-based indexing of textual data G06F16/31) · CPC title