Dynamic resource selection for uploading multi-part data in a dispersed storage network

Patent metadata

Field	Value
Publication number	US-2017155709-A1
Application number	US-201615357670-A
Country	US
Kind code	A1
Filing date	Nov 21, 2016
Priority date	Nov 30, 2015
Publication date	Jun 1, 2017
Grant date	—

Abstract

Methods for use in a dispersed storage network (DSN) to determine appropriate resources for storing data such as large, multi-part data. In various examples, a network control unit obtains status information relating to the DSN. The status information, which may be updated periodically, may include information relating to storage unit utilization and performance levels, network loading and performance levels, data size indicators, etc. Based on the status information, the network control unit maps each of a plurality of sets of encoded data slices of data object to a set of storage units, and generates network control information based on the mapping. The network control information, which may include routing instructions and storage set utilization information, is issued to a network routing unit for use in facilitating storage (e.g., via IP address translation) of differing parts of the data object in the sets of storage units in accordance with the mapping.

First claim

What is claimed is:

1 . A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the DSN having a network routing unit and storage resources including a plurality of sets of storage units, the method comprises:
obtaining status information relating, at least in part, to storage units of the plurality of sets of storage units; mapping each of two or more groups of a plurality of sets of encoded data slices to a set of storage units of the plurality of sets of storage units, including mapping at least a first group of sets of encoded data slices (first group of data) to a first set of storage units of the plurality of sets of storage units, and mapping a second group of sets of encoded data slices (second group of data) to a second set of storage units of the plurality of sets of storage units, the mapping based at least in part on the status information; generating network control information based on the mapping, the network control information including at least one of routing instructions or storage set utilization information for use in facilitating storage of the first group of data and the second group of data; and issuing the network control information to the network routing unit.

2 . The method of claim 1 , wherein the first group of data and the second group of data are generated from differing portions of a common data object.

3 . The method of claim 1 , wherein the first group of data is of higher priority relative to the second group of data, and wherein the status information indicates that the first set of storage units is capable of higher performance as compared to the second set of storage units.

4 . The method of claim 1 , wherein the first group of data is of higher priority relative to the second group of data, and wherein the status information indicates that the first set of storage units is associated with a higher cost of storage relative to the second set of storage units.

5 . The method of claim 1 , wherein the first group of data is approximately the same size as the second group of data.

6 . The method of claim 1 , wherein the network control information facilitates parallel processing of the first group of data and the second group of data by the network routing unit.

7 . The method of claim 1 , wherein obtaining status information relating, at least in part, to the storage units includes at least one of: interpreting a network query, receiving a portion of the status information from a computing device of the DSN, receiving a portion of the status information from at least one of the storage units of the plurality of sets of storage units, receiving the status information from the network routing unit, or interpreting a test result.

8 . The method of claim 1 , wherein the status information includes information relating to at least one of: a utilization level of the storage units, a DSN network loading level, a DSN network performance level, a storage unit access performance level, a data storage reliability level, a data size indicator, a slice size indicator, an accessing entity identifier (ID), or an access profile.

9 . The method of claim 8 , wherein the status information is updated on a periodic basis.

10 . The method of claim 1 , wherein the first set of storage units is affiliated with a first family of Internet Protocol (IP) addresses and the second set of storage units is affiliated with a second family of IP addresses, and wherein the network control information references the first family of IP addresses and the second family of IP addresses.

11 . The method of claim 1 further comprises:
receiving a data object for storage;
dispersed storage error encoding the data object to produce a plurality of sets of encoded data slices including the first group of data and the second group of data, wherein the data object is segmented into a plurality of data segments and wherein each data segment is dispersed storage error encoded into a respective set of encoded data slices; and
routing, by the network routing unit, the first group of data to the first set of storage units and the second group of data to the second set of storage units in accordance with the network control information.

12 . The method of claim 11 , wherein the routing includes Internet Protocol (IP) address translation involving IP addresses affiliated with the first set of storage units and the second set of storage units.

13 . A computer readable storage medium comprises:
at least one memory section that stores operational instructions that, when executed by one or more processing modules of a dispersed storage network (DSN) that include a processor and a memory, causes the one or more processing modules to:
obtain status information relating, at least in part, to storage units of the plurality of sets of storage units;
map, based at least in part on the status information, each of two or more groups of a plurality of sets of encoded data slices to a set of storage units of a plurality of sets of storage units, including mapping at least a first group of sets of encoded data slices (first group of data) to a first set of storage units of the plurality of sets of storage units, and mapping a second group of sets of encoded data slices (second group of data) to a second set of storage units of the plurality of sets of storage units;
generate network control information based on the mapping, the network control information including routing instructions for use in facilitating storage of the first group of data and the second group of data; and
issue the network control information to a network routing unit of the DSN.

14 . The computer readable storage medium of claim 13 , wherein the status information includes information relating to at least one of: a utilization level of the storage units of the first set of storage units and the second set of storage units, a DSN network loading level, a DSN network performance level, a storage unit access performance level, a data storage reliability level, a data size indicator, a slice size indicator, an accessing entity identifier (ID), or an access profile.

15 . The computer readable storage medium of claim 13 further comprises:
the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more processing modules to obtain updated status information on a periodic basis.

16 . A computing device of a group of computing devices of a dispersed storage network (DSN), the DSN having a network routing unit and storage resources including a plurality of sets of storage units, the computing device comprises:
a network interface; a local memory; and a processing module operably coupled to the network interface and the local memory, wherein the processing module operates to:
obtain, via the network interface, status information relating, at least in part, to storage units of the plurality of sets of storage units;
map, based at least in part on the status information, each of two or more groups of a plurality of sets of encoded data slices to a set of storage units of a plurality of sets of storage units, including mapping at least a first group of sets of encoded data slices (first group of data) to a first set of storage units of the plurality of sets of storage units, and mapping a second group of sets of encoded data slices (second group of data) to a second set of storage units of the plurality of sets of storage units;
generate network control information based

Assignees

• IBM

Inventors

• Badanahatti Venkata Giri
• Hegde Harsha

Classifications

• G06F3/0611

  in relation to response time · CPC title

• G06F11/1076

  Parity data used in redundant arrays of independent storages, e.g. in RAID systems · CPC title

• H03M13/3761

  using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F3/0635 Primary

  by changing the path, e.g. traffic rerouting, path reconfiguration · CPC title