Fragment and shuffle erasure coding technique

Patent metadata

Field	Value
Publication number	US-12505230-B2
Application number	US-202217663424-A
Country	US
Kind code	B2
Filing date	May 14, 2022
Priority date	May 14, 2022
Publication date	Dec 23, 2025
Grant date	Dec 23, 2025

Abstract

A fragment and shuffle erasure coding operation is disclosed. In a transformative operation, encrypted fragments are generated from an input file. The transformative operation includes one or more iterations of chunking the file and mixing the chunks to generate a mixed sequence. The mixed sequence is sliced into encrypted fragments. The encrypted fragments are shuffled in a storage system such that physical locations of the encrypted fragments change over time.

First claim

What is claimed is:

1 . A method, comprising:
receiving a file into an erasure coding engine, wherein the file is stored in a storage system; generating, by the erasure coding engine, encrypted fragments from the file in a transformative operation that includes chunking the file at least once into chunks, mixing an order of the chunks for each chunking and forming a sequence, and slicing the sequence into the encrypted fragments; performing a shuffling operation, by the erasure coding engine, to shuffle the encrypted fragments in the storage system to change physical storage locations of at least some of the encrypted fragments; and maintaining a shuffle index that maps logical locations of the encrypted fragments in the storage system to physical locations of the encrypted fragments in the storage system.

2 . The method of claim 1 , wherein the transformative operation includes generating first chunks, mixing the first chunks into a first sequence, generating second chunks from the first chunks in the first sequence, and mixing the second chunks into a second sequence.

3 . The method of claim 2 , wherein the transformative operation includes performing encryption.

4 . The method of claim 2 , wherein each of the encrypted fragments includes a piece from each of the first chunks.

5 . The method of claim 1 , further comprising shuffling the encrypted fragments such that the encrypted fragments are stored on multiple storage nodes of the storage system.

6 . The method of claim 5 , wherein the erasure coding engine operates on a specific node in the storage system.

7 . The method of claim 5 , wherein the erasure coding engine includes multiple instances operating at multiple nodes in the storage system.

8 . The method of claim 1 , further comprising shuffling the encrypted fragments to protect against eavesdropping, traffic analysis, and network pattern recognition.

9 . The method of claim 1 , further comprising performing the shuffling operation after generating the encrypted fragments, after a predetermined amount of time, and/or after a number of data accesses in the storage system.

10 . A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
receiving a file into an erasure coding engine, wherein the file is stored in a storage system; generating, by the erasure coding engine, encrypted fragments from the file in a transformative operation that includes chunking the file at least once into chunks, mixing an order of the chunks for each chunking and forming a sequence, and slicing the sequence into the encrypted fragments; and performing a shuffling operation, by the erasure coding engine, to shuffle the encrypted fragments in the storage system to change physical storage locations of at least some of the encrypted fragments; and performing the shuffling operation after generating the encrypted fragments, after a predetermined amount of time, and/or after a number of data accesses in the storage system.

11 . The non-transitory storage medium of claim 10 , wherein the transformative operation includes generating first chunks, mixing the first chunks into a first sequence, generating second chunks from the first chunks in the first sequence, and mixing the second chunks into a second sequence.

12 . The non-transitory storage medium of claim 11 , wherein the transformative operation includes performing encryption.

13 . The non-transitory storage medium of claim 11 , wherein each of the encrypted fragments includes a piece from each of the first chunks.

14 . The non-transitory storage medium of claim 10 , further comprising shuffling the encrypted fragments such that the encrypted fragments are stored on multiple storage nodes of the storage system.

15 . The non-transitory storage medium of claim 14 , wherein the erasure coding engine operates on a specific node in the storage system.

16 . The non-transitory storage medium of claim 14 , wherein the erasure coding engine includes multiple instances operating at multiple nodes in the storage system.

17 . The non-transitory storage medium of claim 10 , further comprising shuffling the encrypted fragments to protect against eavesdropping, traffic analysis, and network pattern recognition.

18 . The non-transitory storage medium of claim 10 , further comprising maintaining a shuffle index that maps logical locations of the encrypted fragments in the storage system to physical locations of the encrypted fragments in the storage system.

Assignees

• Dell Products Lp

Inventors

• Barnett Alan
• Globin Merry
• Mahony Aidan O
• Keating Matthew

Classifications

• G06F16/116

  Details of conversion of file system types or formats · CPC title

• G06F21/602 Primary

  Providing cryptographic facilities or services · CPC title