Dispersed storage network data manipulation

Patent metadata

Field	Value
Publication number	US-9167277-B2
Application number	US-77609810-A
Country	US
Kind code	B2
Filing date	May 7, 2010
Priority date	Aug 3, 2009
Publication date	Oct 20, 2015
Grant date	Oct 20, 2015

Abstract

A method begins by a processing module receiving data for dispersed storage, wherein the data has an associated user identification (ID), and obtaining a codec flag based on the associated user ID. The codec flag may indicate one or more codec types and a codec execution order. The codec types may include two or more of: a null data manipulation, one or more versions of a data integrity function, one or more versions of a compression function, and/or one or more versions of an encryption function. The method continues with the processing module manipulating the data to produce manipulated data utilizing the one or more codec types in the codec execution order, encoding the manipulated data using an error coding dispersal storage function to produce a plurality of encoded data slices, and sending the slices and codec flag to a plurality of DS storage units for storage.

First claim

What is claimed is:

1. A method for execution by a dispersed storage (DS) processing unit, the method comprises:
receiving a data segment of a plurality of data segments of a data object for dispersed storage, wherein the data object has an associated user identification (ID) of a user;
obtaining a codec flag based on the associated user ID, wherein the codec flag includes a current state field and a codec stack field, in which the current state field is initialized in a null state and in which the codec stack field contains numerical values in execution order to access respective numerical entries in a data manipulation table where the numerical entries correspond to different codec functions for use in manipulating the data segment, and wherein the codec functions listed in the data manipulation table include one or more versions of a data integrity function, one or more versions of a compression function, and one or more versions of an encryption function;
manipulating the data segment using selected codec functions by selecting a first numerical value of the numerical values contained in the codec stack field in the execution order and placing the first numerical value in the current state field to perform a first codec function corresponding to a respective numerical entry in the data manipulation table and further manipulating the data segment by selecting subsequent numerical value or values contained in the codec stack field, one at a time in the execution order, for placing into the current state field to perform a respective codec function, until codec functions indicated in the codec stack field are completed to generate a manipulated data;
encoding the manipulated data using an error coding dispersal storage function to generate a plurality of encoded data slices for storage, wherein a read threshold number of encoded data slices of the plurality of encoded data slices are needed to recover the manipulated data, where the read threshold number of encoded data slices is less than a number corresponding to the plurality of encoded data slices; and
outputting the plurality of encoded data slices and the codec stack field of the codec flag to a plurality of DS storage units for storage therein.

2. The method of claim 1 , wherein the encoding further comprises:
appending the codec stack field of the codec flag to the manipulated data to produce appended manipulated data; and
encoding the appended manipulated data using the error coding dispersal storage function to produce the plurality of encoded data slices.

3. The method of claim 1 , wherein the outputting further comprises:
appending the codec stack field of the codec flag to the plurality of encoded data slices to produce an appended plurality of encoded data slices; and
outputting the appended plurality of encoded data slices to the plurality of DS storage units for storage therein.

4. The method of claim 1 further comprises:
obtaining a second codec flag based on the associated user ID, wherein the codec stack field of the second codec flag indicates a second execution order of codec functions to perform in manipulating the plurality of encoded data slices;
manipulating the plurality of encoded data slices using one or more codec functions corresponding to the codec stack field of the second codec flag in the second codec execution order to generate manipulated encoded data slices; and
outputting the plurality of encoded data slices as the manipulated encoded data slices, along with the codec stack field of the second codec flag to the plurality of DS storage units for storage therein.

5. The method of claim 1 , wherein the codec stack field of the codec flag further has repeat numerical values to perform a same codec function, but at a different period in the execution order.

6. The method of claim 1 , wherein the obtaining further comprises:
identifying a vault identity (ID) based on the associated user ID;
identifying a registry descriptor based on the vault ID; and
accessing the registry descriptor from a plurality of registry descriptors to retrieve vault information, wherein the vault information includes a codec stack for the codec stack field and identifies the data manipulation table.

7. A method for execution by a dispersed storage (DS) processing unit, the method comprises:
providing access to at least a threshold number of a plurality of encoded data slices of a data segment of a plurality of data segments of a data object stored within a plurality of DS storage units, wherein the threshold number of encoded data slices of the plurality of encoded data slices are needed to recover the data segment, the threshold number of encoded data slices being less than the plurality of encoded data slices;
receiving a codec stack field of a codec flag and the threshold number of encoded data slices of the plurality of encoded data slices from the plurality of DS storage units, wherein the codec stack field contains numerical values in execution order to access respective numeric entries in a data manipulation table where the numerical entries correspond to different codec functions for use in recovering the data segment from the threshold number of encoded data slices, and wherein the codec functions listed in the data manipulation table include one or more versions of a data integrity function, one or more versions of a compression function, and one or more versions of an encryption function;
decoding the threshold number of encoded data slices using an error coding dispersal storage function to produce manipulated data; and
de-manipulating the manipulated data using selected codec functions by selecting a first numerical value of the numerical values contained in the codec stack field in the execution order and placing the first numerical value in a current state field of the codec flag to perform a first codec function corresponding to a respective numerical entry in the data manipulation table and further de-manipulating the manipulated data by selecting subsequent numerical value or values contained in the codec stack field, one at a time in the execution order, for placing into the current state field to perform a respective codec function, until codec functions indicated in the codec stack field are completed to recover the data segment.

8. The method of claim 7 further comprises:
receiving the threshold number of encoded data slices from the plurality of DS storage units;
decoding the threshold number of encoded data slices using the error coding dispersal storage function to produce appended manipulated data; and
separating the appended manipulated data to produce the manipulated data and the codec stack field.

9. The method of claim 7 further comprises:
receiving an appended threshold number of encoded data slices from the plurality of DS storage units; and
separating the appended encoded data slices to produce the threshold number of encoded data slices and the codec stack field.

10. The method of claim 7 further comprises:
receiving the threshold number of encoded data slices, the codec stack field, and a second codec field from the plurality of DS storage units, wherein the second codec stack field indicates a second execution order of codec functions to perform in de-manipulating the manipulated data; and
de-manipulating at least one of the plurality of encoded data slices using the second execution order.

11. A dispersed storage (DS) processing unit comprises:
a network interface; and
a processing module operable to:
obtain a data segment of a plurality of data segments of a data object for dispersed storage, wherein the data object has an associated user identification (ID) of a use

Assignees

• Cilfone Bart
• Leggette Wesley
• Resch Jason K
• Cleversafe Inc

Inventors

• Cilfone Bart
• Leggette Wesley
• Resch Jason K

Classifications

• H04N21/25875

  involving end-user authentication (restricting access to computer systems by authenticating users using a predetermined code G06F21/33; arrangements for secret or secure communication including means for verifying the identity or authority of a user of the system H04L9/32; networks authentication protocols H04L63/08; authentication in wireless network security H04W12/06) · CPC title

• H04N21/2747

  Remote storage of video programmes received via the downstream path, e.g. from the server · CPC title

• G06F11/1076

  Parity data used in redundant arrays of independent storages, e.g. in RAID systems · CPC title

• H04N21/23116 Primary

  involving data replication, e.g. over plural servers (synchronization of replicated data G06F11/1658; error detection or correction by means of data replication G06F11/2053; replication in distributed file systems G06F16/10; replication in distributed file systems G06F16/27; replication or mirroring of data in data networks H04L67/1095) · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title