Hybrid message-based scheduling technique

Patent metadata

Field	Value
Publication number	US-2016246742-A1
Application number	US-201615051057-A
Country	US
Kind code	A1
Filing date	Feb 23, 2016
Priority date	Feb 25, 2015
Publication date	Aug 25, 2016
Grant date	—

Abstract

A hybrid message-based scheduling technique efficiently load balances a storage I/O stack partitioned into one or more non-blocking (i.e., free-running) messaging kernel (MK) threads that execute non-blocking message handlers (i.e., non-blocking services) and one or more operating system kernel blocking threads that execute blocking services. The technique combines the blocking and non-blocking services within a single coherent extended programming environment. The messaging kernel (MK) operates on processors apart from the operating system kernel that are allocated from a predetermined number of logical processors (i.e., hyper-threads) for use by an MK scheduler to schedule the non-blocking services within storage I/O stack as well as allocate a remaining number of logical processors for use by the blocking services. In addition, the technique provides a variation on a synchronization primitive that allows signaling between the two types of services (i.e., non-blocking and blocking) within the extended programming environment.

First claim

What is claimed is:

1 . A method comprising:
receiving an input/output (I/O) request at a storage system, the I/O request processed by a storage I/O stack executing on the storage system, wherein the storage I/O stack is divided into blocking and non-blocking partitions; associating the I/O request with one or more messages; and processing the messages in one or more non-blocking services executing within the non-blocking partition and in one or more blocking services executing within the blocking partition, such that the I/O request is processed by the one or more blocking services interacting with the one or more non-blocking services of the storage system, wherein each message includes a context for processing the respective message.

2 . The method of claim 1 wherein the non-blocking partition executes on a first logical processor of the storage system and the blocking partition executes on a second logical processor of the storage system, wherein the first and second logical processors share common compute resources, and wherein the first and second logical processors have s different register files.

3 . The method of claim 2 further comprising:
processing a first message for performing a first subset of work to process the I/O request in a first blocking service executing within the blocking partition of the storage I/O stack;
queuing, by the first blocking service, a second message for performing a second subset of work to process the I/O request in a first non-blocking service executing within the non-blocking partition of the storage I/O stack;
suspending execution of the first blocking service; and
in response to suspending execution of the first blocking service, stealing cycles of the common compute resources from the second logical processor to process the second subset of work.

4 . The method of claim 3 wherein suspending execution of the first blocking service further comprises:
obtaining a spin-lock on a condition that is synchronized using a condition variable; and
in response to the condition not being satisfied, suspending execution of the first blocking service.

5 . The method of claim 4 further comprising:
signaling, by the first non-blocking service, resumption of execution of the first blocking service.

6 . The method of claim 5 wherein the first blocking service executes an administrative function.

7 . The method of claim 3 wherein a Portable Operating System Interface (POSIX) semaphore wait operation suspends execution of the first blocking service.

8 . The method of claim 5 further comprising:
processing the first subset of work such that the second logical processor prevents stealing of cycles of the common compute resources.

9 . The method of claim 8 further comprising:
executing, in a second non-blocking service, a third subset of work of the I/O request concurrently with the second subset of work, the second non-blocking service executing in the non-blocking
partition of the I/O stack on a third logical processor.

10 . A method comprising:
receiving an input/output (I/O) request at a system, the I/O request processed by an I/O stack executing at a node of the system, wherein the I/O stack is divided into block and non-blocking partitions, wherein the non-blocking partition executes on a first logical processor of the node and the blocking partition executes on a second logical process of the node, wherein the first and second logical processors share common compute resources, wherein the first and second logical processors have different register files; associating the I/O request with a plurality of messages for processing the I/O request; processing a first message for performing a first subset of work to process the I/O request in a first service executing within the blocking partition of the I/O stack; queuing, by the first service, a second message for performing a second subset of work to process the I/O request in a second service executing within the non-blocking partition of the I/O stack; suspending execution of the first service; and in response to suspending execution of the first service, stealing cycles of the common compute resources from the second logical processor to process the second subset of work.

11 . A system comprising:
a storage system having a memory connected to a processor via a bus; and a storage I/O stack executing on the processor of the storage system, the storage I/O stack divided into blocking and non-blocking partitions, the storage I/O stack when executed operable to:
receive an I/O request;
associate the I/O request with one or more messages for processing the I/O request; and
process the messages in one or more non-blocking services executing within the non-blocking partition such that the I/O request is processed by the storage system, wherein each message includes a context for processing the respective message.

12 . The system of claim 11 wherein the non-blocking partition executes on a first core of the processor and the blocking partition executes on a second core of the processor, wherein the first and second cores share common compute resources, and wherein the first and second cores have different register files.

13 . The system of claim 11 wherein the storage I/O stack when executed is further operable to:
process a first message for performing a first subset of work to process the I/O request in a first service executing within the blocking partition of the storage I/O stack;
queue, by the first service, a second message for performing a second subset of work to process the I/O request in a second service executing within the non-blocking partition of the I/O stack;
suspend execution of the first service; and
in response to suspending execution of the first service, steal cycles of the common compute resource from the second logical processor to process the second subset of work.

14 . The system of claim 13 wherein the storage I/O stack when executed to suspend execution of the first service is further operable to:
obtain a spin-lock on a condition that is synchronized using a condition variable; and
in response to the condition not being satisfied, suspend execution of the first service.

15 . The system of claim 14 wherein the storage I/O stack when executed is further operable to:
signal, by the second service, resumption of execution of the first service.

16 . The system of claim 15 wherein the first service executes an administrative function.

17 . The system of claim 13 wherein a Portable Operating System Interface (POSIX) semaphore wait operation suspends execution of the first service.

18 . The system of claim 15 wherein the storage I/O stack when executed is further operable to:
process the first subset of work such that the second logical process prevents stealing of cycles of the common compute resources.

19 . The system of claim 18 wherein the storage I/O stack when executed is further operable to:
perform, in a third service, a third subset of work of the I/O request concurrently with the second subset of work, the third service executing in the non-blocking partition of the storage I/O stack on a third core of the processor.

20 . The system of claim 19 wherein the storage I/O stack when executed is further operable to:
poll an I/O device for I/O completion, wherein the third core of the processor is dedicated to polling the I/O device.

Assignees

• Netapp Inc

Inventors

• Kimmel Jeffrey S

Classifications

• G06F3/061 Primary

  Improving I/O performance · CPC title

• G06F13/36 Primary

  for access to common bus or bus system · CPC title

• G06F9/546

  Message passing systems or structures, e.g. queues · CPC title

• G06F9/5083

  Techniques for rebalancing the load in a distributed system · CPC title

• G06F3/0658

  Controller construction arrangements · CPC title