Techniques for increasing storage system performance in processor-bound workloads with large working sets and poor spatial locality

Patent metadata

Field	Value
Publication number	US-10235203-B1
Application number	US-201414230118-A
Country	US
Kind code	B1
Filing date	Mar 31, 2014
Priority date	Mar 31, 2014
Publication date	Mar 19, 2019
Grant date	Mar 19, 2019

Abstract

An improved technique involves processing a workflow in stages, and processing all requests in a queue for a given stage before moving onto the next stage. Along these lines, each request received by a storage processor is assigned to a core and placed in a first queue for that core. Within that core, a single system thread executes first instructions for a task, e.g., checking the storage cache for the requested data from a request, and then transfers the request to a second queue. Rather than perform additional tasks to completely satisfy the request, however, the thread executes the first instructions for a prespecified number of requests in the first queue. Only when the thread has executed instructions for the prespecified number of requests, the thread begins execution of second instructions for requests in the second queue, and work on the next task begins.

First claim

What is claimed is:

1. In a data storage system including a storage processor having a set of cores, a method of processing read/write requests received from requestors coupled to ports of the data storage system by a network, each read/write request being a storage read/write request directed to a respective addressable unit of secondary storage provided by the data storage system, the method comprising, for each of the set of cores of the storage processor:
receiving a set of the read/write requests from the ports, the set of the read/write requests including multiple read/write requests, and assigning each of the set of the read/write requests to be serviced by a process run by that core, the process containing instructions and data and being arranged, upon execution, to carry out a set of tasks performed sequentially at corresponding stages of a workflow by which the set of the read/write requests are fulfilled;
maintaining, by that core, a plurality of queues corresponding to respective stages of the workflow;
storing, by that core, the set of the read/write requests in a first queue of the queues for that core, the first queue being arranged to temporarily hold read/write requests being serviced by the process at a first stage of the workflow;
for each of the set of the read/write requests in the first queue, executing, by that core, first instructions of the process to carry out a first task of the set of tasks at the first stage of the workflow; and
transferring, by that core after the execution of the first instructions for all of the set of the read/write requests in the first queue, only selected ones of the set of the read/write requests from the first queue to a second queue of the queues for that core, the second queue being arranged to temporarily hold the selected ones of the set of the read/write requests to be serviced by the process at a second stage of the workflow,
wherein the queues are ordered according to a predetermined order to process the set of the read/write requests at the corresponding stages of the workflow;
wherein the queues include a high-priority queue for storing read/write requests at a corresponding high-priority stage of the workflow;
wherein the predetermined order is a non-linear order in which the high-priority queue is serviced after each servicing of any lower-priority queues, and wherein the predetermined order includes:
first, processing requests of a normal queue to perform a storage cache lookup for a first burst size of read/write requests in the normal queue;
second, performing a return task for each read/write request in the high-priority queue until the high-priority queue is empty;
third, performing a backend storage lookup for a second burst size of read/write requests in a backend queue;
fourth, performing the return task for each read/write request in the high-priority queue until the high-priority queue is empty;
fifth, performing a low-priority task for a request in a low-priority queue; and
sixth, performing the return task for each read/write request in the high-priority queue until the high-priority queue is empty.

2. A method as in claim 1 , wherein the first task is a storage cache access to a storage cache located either in main memory of the storage processor or in a secondary storage device of the data storage system;
wherein the execution of the first instructions for a first read/write request stored in the first queue results in a storage cache hit; and
wherein the return task comprises returning the result of the storage cache hit to a corresponding requestor of the first read/write request and excluding the first read/write request from the selected ones of the set of the read/write requests transferred to the second queue.

3. A method as in claim 1 , wherein the first task is a storage cache access to a storage cache located either in main memory of the storage processor or it a secondary storage device of the data storage system;
wherein the execution of the first instructions for plural read/write requests stored in the first queue results in respective storage cache misses; the method further comprising including the plural read/write requests in the selected ones of the set of the read/write requests transferred to the second queue in the transferring step.

4. A method as in claim 1 , wherein each of the queues is arranged to temporarily hold respective read/write requests being serviced by the process at respective stages of the workflow;
wherein one of the queues is a final queue for a final stage of the workflow, the final stage including the return task for a particular read/write request of returning a response to a requestor from which the particular read/write request was received; and
wherein the method further comprises, for each of a number of the read/write requests stored in the final queue:
executing, by that core, final instructions of the process to carry out the return task at the final stage of the workflow for that read/write request,
the return task including returning a result for that read/write request to the requestor from which the particular read/write request was received.

5. A method as in claim 1 , wherein the first task is followed by a second task of performing a backend storage access to secondary storage devices of the data storage system; and
wherein the method further comprises, for each of the selected ones of the set of the read/write requests in the second queue:
executing, by that core, second instructions of the process for that read/write request to carry out the backend storage access at the second stage of the workflow;
after executing the second instructions, receiving a signal that the backend storage access has completed; and
after receiving the signal, transferring that read/write request to a final queue of the queues, the final queue used to temporarily store read/write requests for a final stage of the workflow, the final stage including the return task for each read/write request of returning a response to a requestor from which that read/write request was received.

6. A method as in claim 1 , further comprising: providing, to each queue of the queues, a burst size specifying a plural number of read/write requests in that queue for which instructions to carry out a corresponding task are to be executed before executing instructions for read/write requests in a next queue according to the predetermined order; and
executing, at each stage of the workflow, instructions for the plural number of read/write requests in the queue to which that stage corresponds, the plural number of read/write requests being no greater than the burst size for the queue to which that stage corresponds,
wherein the first burst size is the burst size for the normal queue, and the second burst size is the burst size for the backend queue.

7. A method as in claim 6 , wherein providing the burst size to each queue includes assigning an infinite burst size to the high-priority queue, the infinite burst size indicating that instructions for all read/write requests in the high-priority queue are to be executed before executing instructions of read/write requests in lower-priority ones of the queues according to the predetermined order.

8. A method as in claim 1 , wherein each core includes a processor cache having a multilevel arrangement of memory including a level one cache being private to that core;
wherein the process run by each core is a single-threaded process that includes a working set of instructions and data, the working set being substantially larger than the level one cache of that core; and
wherein the method further comprises forming the set of tasks so that instructions and data for each of the set of tasks is smalle

Assignees

• Emc Corp
• Emc Ip Holding Co Llc

Inventors

• Cummins Daniel
• Harvey David W
• Morley Steve

Classifications

• G06F12/084

  with a shared cache · CPC title

• G06F12/0871

  Allocation or management of cache space · CPC title

• G06F2212/1021

  Hit rate improvement · CPC title

• G06F9/4881 Primary

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title

• G06F2212/263

  Network storage, e.g. SAN or NAS · CPC title