Site-based search affinity

Patent metadata

Field	Value
Publication number	US-9130971-B2
Application number	US-201414266812-A
Country	US
Kind code	B2
Filing date	Apr 30, 2014
Priority date	May 15, 2012
Publication date	Sep 8, 2015
Grant date	Sep 8, 2015

Abstract

According to various embodiments, techniques are described for managing data within a multi-site clustered data intake and query system. A data intake and query system as described herein generally refers to a system for collecting, retrieving, and analyzing data. In this context, a clustered data intake and query system generally refers to a system environment that is configured to provide data redundancy and other features that improve the availability of data stored by the system. For example, a clustered data intake and query system may be configured to store multiple copies of data stored by the system across multiple components such that recovery from a failure of one or more of the components is possible by using copies of the data stored elsewhere in the cluster.

First claim

What is claimed is:

1. A method comprising:
receiving, at an indexer, a set of search affinity identifiers;
wherein each search affinity identifier of the set of search affinity identifiers indicates, for a particular subset of data accessible to the indexer and for a particular site of a plurality of sites from which a query may originate, whether the indexer has primary responsibility for responding to queries originating from the particular site based on data from the particular subset of data;
wherein each site of the plurality of sites represents a user-specified grouping of one or more computing resources corresponding to a particular geographic location;
receiving, from a first search head located at a first site, (i) a first query to search a subset of data accessible to the indexer, and (ii) a first site identifier identifying the first site at which the first search head is located;
determining, based on both the first site identifier and a particular search affinity identifier of the set of search affinity identifiers, that the indexer is to respond to the first query with a result from searching the subset of data;
in response to determining that the indexer is to respond to the first query, sending, to the first search head, the result from searching the subset of data.

2. The method of claim 1 , wherein determining that the indexer is to respond to the first query with a result from searching the subset of data includes determining that the indexer has primary responsibility for responding to queries for the subset of data for the first site.

3. The method of claim 1 , further comprising:
receiving, at the indexer, a second query from a second search head to search the subset of data, the second query including a second site identifier identifying a second site at which the second search head is located;
determining, based on both the second site identifier and the particular search affinity identifier of the set of search affinity identifiers, that the indexer is not to respond to the second query with a result from searching the subset of data.

4. The method of claim 1 , wherein the particular search affinity identifier is a bitmask, and wherein each digit of the bitmask represents a particular site of the plurality of sites from which a query may originate.

5. The method of claim 1 , further comprising:
receiving, at the indexer, a second query from a second search head to search the subset of data, the second query including a second site identifier identifying a second site at which the second search head is located;
determining, based on both the second site identifier and the particular search affinity identifier of the set of search affinity identifiers, that the indexer is not to respond to the second query with a result from searching the subset of data;
wherein the first query and the second query are identical.

6. The method of claim 1 , further comprising:
receiving, at the indexer, a second query from a second search head to search the subset of data, the second query including a second site identifier identifying a second site at which the second search head is located;
determining, based on both the second site identifier and the particular search affinity identifier of the set of search affinity identifiers, that the indexer is not to respond to the second query with a result from searching the subset of data;
wherein the first query and the second query are different.

7. The method of claim 1 , further comprising:
receiving, at the indexer, raw data;
separating the raw data into a plurality of events included in the subset of data;
determining, for each event in the plurality of events, a time stamp; and
storing the subset of data in a data store.

8. The method of claim 1 , further comprising:
receiving, at the indexer, raw data;
separating the raw data into a plurality of events included in the subset of data;
storing the subset of data in a data store;
identifying a replication factor that indicates a number of times that the subset of data is to be replicated; and
sending the subset of data to a number of other indexers, wherein the number corresponds to the replication factor.

9. The method of claim 1 , further comprising:
receiving, at the indexer, raw data;
separating the raw data into a plurality of events included in the subset of data;
storing the subset of data in a data store;
identifying a site replication factor that indicates a number of sites at which the subset of data is to be replicated; and
sending the subset of data to second indexers located at the number of sites.

10. The method of claim 1 , further comprising:
wherein the set of search affinity identifiers is associated with a first generation identifier;
receiving, at the indexer, a second set of search affinity identifiers associated with a second generation identifier.

11. The method of claim 1 , further comprising:
wherein the indexer stores a plurality of sets of search affinity identifiers, and wherein each set of search affinity identifiers of the plurality of sets of search affinity identifiers is associated with a generation identifier;
receiving, from the first search head, a particular generation identifier identifying a particular set of search affinity identifiers of the plurality of sets of search affinity identifiers.

12. One or more non-transitory computer-readable storage media, storing software instructions, which when executed by one or more processors cause performance of steps of:
receiving, at an indexer, a set of search affinity identifiers;
wherein each search affinity identifier of the set of search affinity identifiers indicates, for a particular subset of data accessible to the indexer and for a particular site of a plurality of sites from which a query may originate, whether the indexer has primary responsibility for responding to queries originating from the particular site based on data from the particular subset of data;
wherein each site of the plurality of sites represents a user-specified grouping of one or more computing resources corresponding to a particular geographic location;
receiving, from a first search head located at a first site, (i) a first query to search a subset of data accessible to the indexer, and (ii) a first site identifier identifying the first site at which the first search head is located;
determining, based on both the first site identifier and a particular search affinity identifier of the set of search affinity identifiers, that the indexer is to respond to the first query with a result from searching the subset of data;
in response to determining that the indexer is to respond to the first query, sending, to the first search head, the result from searching the subset of data.

13. The one or more non-transitory computer-readable storage media of claim 12 , wherein determining that the indexer is to respond to the first query with a result from searching the subset of data includes determining that the indexer has primary responsibility for responding to queries for the subset of data for the first site.

14. The one or more non-transitory computer-readable storage media of claim 12 , wherein the instructions, when executed by the one or more computing devices, further cause performance of:
receiving, at the indexer, a second query from a second search head to search the subset of data, the second query including a second site identifier identifying a second site at which the second search head is located;
determining, based on both the second site identifier and the particular search affinity identifier of the set of

Assignees

• Splunk Inc
• Splunk Inc

Inventors

• Vasan Sundar Rengarajan
• Blank Jr Mitchell Neuman
• Patel Vishal
• Xu Da
• Gopalan Rama

Classifications

• G06F11/2094

  Redundant storage or storage space (G06F11/2056 takes precedence) · CPC title

• G06F3/0617

  in relation to availability · CPC title

• G06F16/29

  Geographical information databases · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F16/275

  Synchronous replication · CPC title