Network system to distribute chunks across multiple physical nodes with disk support for object storage

Patent metadata

Field	Value
Publication number	US-9852151-B1
Application number	US-201414552387-A
Country	US
Kind code	B1
Filing date	Nov 24, 2014
Priority date	Jun 28, 2013
Publication date	Dec 26, 2017
Grant date	Dec 26, 2017

Abstract

A method of storing a file in a storage system that includes a plurality of memory-storage hosts includes: specifying a unique chunk identifier for a memory chunk included in the file; specifying a hash mapping to identify one or more storage locations for the chunk identifier, each storage location corresponding to a portion of a memory-storage host; providing metadata corresponding to the hash mapping to the memory-storage hosts; and storing the memory chuck at the one or more storage locations identified by the hash mapping by providing the chunk identifier to one or more memory-storage hosts corresponding to the identified one or more storage locations, the one or more memory-storage hosts implementing the hash mapping to store the memory chunk at the identified one or more locations.

First claim

What is claimed is:

1. A method of storing a file in a storage system that includes a plurality of memory-storage hosts, the method comprising:
specifying a unique chunk identifier for a memory chunk included in the file;
specifying a hash mapping to identify a plurality of storage locations for the chunk identifier, each memory-storage host including a corresponding disk-storage unit, each storage location corresponding to a portion of a disk-storage unit included in a memory-storage host, the hash mapping including host-identification outputs and host-storage outputs, the host-identification outputs identifying a plurality of the memory-storage hosts for storing the memory chunk, and the host-storage outputs identifying one or more storage locations at each identified memory-storage host;
providing metadata corresponding to a first portion of the hash mapping to a memory-management unit for the storage system, the metadata corresponding to the first portion of the hash mapping being used to access the first portion of the hash mapping, and the memory management unit being configured to implement the first portion of the hash mapping by determining the host-identification outputs from the hash mapping;
providing metadata corresponding to a second portion of the hash mapping to a disk-storage interface of a corresponding disk-storage unit for each memory-storage host of the plurality of memory storage hosts, the metadata corresponding to the second portion of the hash mapping being used to access the second portion of the hash mapping, and the disk-storage interface being configured to implement the second portion of the hash mapping by determining the host-storage outputs that identify storage locations at that memory-storage host;
maintaining, at each memory-storage host of the plurality of memory-storage hosts, a shared memory with the memory-management unit, the shared memory including one or more buffers for accessing storage locations of that memory-storage host by the memory-management unit; and
storing the memory chunk at the storage locations identified by the hash mapping by using the memory-management unit to determine the host-identification outputs from the hash mapping and to provide the chunk identifier to the shared memory of each identified memory-storage host corresponding to the host-identification outputs, and by using each identified memory-storage host to determine its corresponding host-storage outputs from the hash mapping and to store the memory chunk at the identified one or more storage locations corresponding to its host-storage outputs via its shared memory with the memory-management unit.

2. The method of claim 1 , wherein each memory-storage host maintains a cache memory at its corresponding disk-storage interface, the cache memory including values for the metadata corresponding to the second portion of the hash mapping.

3. The method of claim 1 , wherein each memory-storage host implements the second portion of the hash mapping in firmware at its corresponding disk-storage interface.

4. The method of claim 1 , wherein each disk-storage unit is accessible by a addressing a block size and a block offset, and each disk-storage interface implements the second portion of the hash mapping in firmware to relate the unique chunk identifier to a corresponding block size and block offset.

5. The method of claim 1 , wherein the unique chunk identifier includes a node ID (identification number) to identify a memory-storage host, a LUN ID to identify a Logical Unit Number (LUN) within the memory-storage host, and a Chunk ID to identify a chunk of memory with the LUN, each disk-storage interface implementing the second portion of the hash mapping in response to storage requests from the memory-storage hosts of the storage system.

6. The method of claim 1 , wherein the file includes a plurality of memory chunks that have a uniform size given by an integral multiple of a memory block size associated with the storage system.

7. The method of claim 1 , further comprising:
updating the metadata corresponding to the second portion of the hash mapping when a memory-storage host is added to the system or deleted from the system; and
providing the updated metadata corresponding to the second portion of the hash mapping to the memory-storage hosts for updating the implementation of the second portion of the hash mapping at the disk-storage interfaces of the disk-storage units.

8. The method of claim 1 , wherein the shared memory of a memory storage host of the plurality of memory storage hosts includes an input/output queue configured to implement reading and writing operations between a controller of the memory management unit and a controller of the memory-storage host.

9. A non-transitory computer-readable medium that stores a computer program for storing a file in a storage system that includes a plurality of memory-storage hosts, the computer program including instructions that, when executed by at least one computer, cause the at least one computer to perform operations comprising:
specifying a unique chunk identifier for a memory chunk included in the file;
specifying a hash mapping to identify a plurality of storage locations for the chunk identifier, each memory-storage host including a corresponding disk-storage unit, and each storage location corresponding to a portion of a disk-storage unit included in a memory-storage host, the hash mapping including host-identification outputs and host-storage outputs, the host-identification outputs identifying a plurality of the memory-storage hosts for storing the memory chunk, and the host-storage outputs identifying one or more storage locations at each identified memory-storage host;
providing metadata corresponding to a first portion of the hash mapping to a memory-management unit for the storage system, the metadata corresponding to the first portion of the hash mapping being used to access the first portion of the hash mapping, and the memory management unit being configured to implement the first portion of the hash mapping by determining the host-identification outputs from the hash mapping;
providing metadata corresponding to a second portion of the hash mapping to a disk-storage interface of a corresponding disk-storage unit for each memory-storage host of the plurality of memory storage hosts, the metadata corresponding to the second portion of the hash mapping being used to access the second portion of the hash mapping, and the disk-storage interface being configured to implement the second portion of the hash mapping by determining the host-storage outputs that identify storage locations at that memory-storage host;
maintaining, at each memory-storage host of the plurality of memory-storage hosts, a shared memory with the memory-management unit, the shared memory including one or more buffers for accessing storage locations of that memory-storage host by the memory-management unit; and
storing the memory chunk at the storage locations identified by the hash mapping by using the memory-management unit to determine the host-identification outputs from the hash mapping and to provide the chunk identifier to the shared memory of each identified memory-storage host corresponding to the host-identification outputs, and by using each identified memory-storage host to determine its corresponding host-storage outputs from the hash mapping and to store the memory chunk at the identified one or more storage locations corresponding to its host-storage outputs via its shared memory with the memory-management unit.

10. The non-transitory computer-readable medium of claim 9 , wherein
each memory-storage host maintains a cache memory at its corresponding disk-stor

Assignees

• Sanmina Corp

Inventors

• Youngworth Christopher

Classifications

• G06F3/0665

  at area level, e.g. provisioning of virtual or logical volumes · CPC title

• G06F16/182 Primary

  Distributed file systems · CPC title

• G06F3/0689 Primary

  Disk arrays, e.g. RAID, JBOD · CPC title

• G06F3/0619

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F3/064

  Management of blocks · CPC title