Dynamic search partitioning

Patent metadata

Field	Value
Publication number	US-10289603-B2
Application number	US-201615362503-A
Country	US
Kind code	B2
Filing date	Nov 28, 2016
Priority date	Oct 12, 2012
Publication date	May 14, 2019
Grant date	May 14, 2019

Abstract

A system can monitor data usage, including an amount of searchable data used and/or a rate at which the searchable data is manipulated, on a storage allocation in a networked environment. The storage allocation can have a quantity/number of partitions, including at least one partition, configured to store the searchable data. The system can detect that the data usage is beyond a specified threshold and then based at least in part on factors such as network traffic, CPU usage, and/or data usage, the system can modify the storage allocation to increase or decrease a size of the partition and/or the quantity of partitions. Network traffic for the storage allocation can be directed away from the portion of the storage allocation being modified. When modifying the storage allocation is complete, the network traffic can be directed to the modified portion of the storage allocation.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
monitoring usage of data for a storage allocation in a networked environment;
determining that network traffic, associated with the storage allocation and corresponding to a search query type, satisfies a modification criterion corresponding to a search query type complexity associated with latency and at least one of a volume of the network traffic or a rate of manipulation of data in the network traffic;
modifying a size of at least one partition of the storage allocation based on the network traffic satisfying the modification criterion;
detecting that a size of a search index, for the at least one partition of the storage allocation, exceeds a specified index size threshold; and
updating the search index.

2. The computer-implemented method of claim 1 , further comprising:
rebuilding the search index for the at least one partition of the storage allocation to reflect the modifying the size of the at least one partition.

3. The computer-implemented method of claim 1 , further comprising:
determining whether the data usage on at least one partition of a set of partitions included in the storage allocation exceeds a specified threshold,
wherein the data usage includes at least one of an amount of data being stored on the storage allocation or a rate at which data is being manipulated on the storage allocation.

4. The computer-implemented method of claim 3 , wherein the specified threshold includes at least one of a specified amount threshold or a specified rate threshold, and wherein the data usage exceeds the specified threshold when there is an occurrence of at least one of the amount of data being stored exceeding the specified amount threshold or the rate at which data is being manipulated exceeding the specified rate threshold.

5. The computer-implemented method of claim 1 , further comprising:
determining that an amount of network traffic directed to the storage allocation is above a specified traffic threshold; and
modifying the storage allocation based on the amount of network traffic.

6. The computer-implemented method of claim 5 ,
wherein the network traffic includes search query traffic for searching data stored on the storage allocation.

7. The computer-implemented method of claim 1 , wherein the modifying the size of the at least one partition further comprises:
increasing the size of the at least one partition based on the size of the at least one partition being below a maximum partition size threshold.

8. The computer-implemented method of claim 1 , wherein the modifying the size of the at least one partition includes decreasing the size of the at least one partition, and further comprising:
decreasing a number of partitions in a set of partitions associated with the storage allocation based on the number of the partitions in the set of partitions being greater than two partitions.

9. The computer-implemented method of claim 1 , further comprising:
determining a CPU usage of the storage allocation, wherein the modifying the size is based on at least one of the data usage on the storage allocation or the determined CPU usage of the storage allocation.

10. The computer-implemented method of claim 1 , further comprising:
modifying a configuration of the storage allocation based on at least one of data associated with the data usage or an indication of input by a user.

11. The computer-implemented method of claim 1 , further comprising:
determining a time to perform the modifying of the size based at least on resources available to the storage allocation.

12. The computer-implemented method of claim 1 , further comprising:
directing the network traffic away from a portion of the storage allocation associated with the modifying.

13. The computer-implemented method of claim 12 , further comprising:
determining that the modifying is complete; and
directing the network traffic to the portion of the storage allocation associated with the modifying when the modifying is complete.

14. The computer-implemented method of claim 1 , wherein the storage allocation includes a number of sets of partitions, each set of partitions associated with a different account.

15. The computer-implemented method of claim 1 , further comprising:
determining that the size of the first partition is at the maximum partition size threshold; and
adding at least an additional partition.

16. A system comprising:
a storage allocation in a networked environment having a plurality of partitions;
at least one processor; and
a memory device including instructions that, when executed by the at least one processor, cause the system to:
monitor usage of data for the storage allocation;
determine that network traffic, associated with the storage allocation and corresponding to a search query type, satisfies a modification criterion corresponding to a search query type complexity associated with latency and at least one of a volume of the network traffic or a rate of manipulation of data in the network traffic;
modify a size of at least one partition of the plurality of partitions based on the network traffic satisfying the modification criterion;
detect that a size of a search index, for the at least one partition of the plurality of partitions, exceeds a specified index size threshold; and
update the search index.

17. The system of claim 16 , wherein the instructions, when executed by the at least one processor, further cause the system to:
determine whether the data usage on at least one partition of a set of partitions associated with the storage allocation exceeds a specified threshold,
wherein the data usage includes at least one of an amount of data being stored on the storage allocation or a rate at which data is being manipulated on the storage allocation.

18. A non-transitory computer-readable storage medium including instructions, which when executed by a processor of a computing system cause the computing system to:
monitor usage of data for a storage allocation in a networked environment;
determine that network traffic, associated with the storage allocation and corresponding to a search query type, satisfies a modification criterion corresponding to a search query type complexity associated with latency and at least one of a volume of the network traffic or a rate of manipulation of data in the network traffic;
modify a size of at least one partition of the plurality of partitions based on the network traffic satisfying the modification criterion;
detect that a size of a search index, for the at least one partition of the plurality of partitions, exceeds a specified index size threshold; and
update the search index.

Assignees

• A9 Com Inc
• Amazon Tech Inc

Inventors

• Goldberg Jonathan Michael
• Makhani Asif Mansoor Ali
• Nwokah Ekechi Karl Edozle

Classifications

• G06F16/1827 Primary

  Management specifically adapted to NAS (management of storage area networks [SAN] G06F3/067) · CPC title

• G06F3/0631

  by allocating resources to storage systems · CPC title

• G06F15/167 Primary

  using a common memory, e.g. mailbox · CPC title