Systems and Methods for Uploading Streamed Objects to a Cloud Storage System

Patent metadata

Field	Value
Publication number	US-2017286698-A1
Application number	US-201715477063-A
Country	US
Kind code	A1
Filing date	Apr 1, 2017
Priority date	Apr 1, 2016
Publication date	Oct 5, 2017
Grant date	—

Abstract

Methods in a cloud object store facilitate strong data encryption, customer-management of object (encryption) keys, reductions in latency, globally-distributed object storage, and handling of streamed uploads. A method for encrypting objects stored in a cloud includes encrypting each object with a unique encryption (object) key. The plaintext object keys are generated in advance of uploads. The plaintext object keys can be stored in an object database in the cloud. Alternatively, the plaintext object keys can be provided to a customer's HSM, encrypted, and returned to the cloud, such that encrypted object keys, encrypted by the customer, are stored in the cloud. The cloud can alternatively encrypt the customer's object keys with a master key for the customer, which is then encrypted by the customer's HSM before being stored in the cloud. Proxies are also deployed for efficiently communicating with customer security modules.

First claim

We claim:

1 . A method for storing digital objects in an object storage system, said method comprising:
establishing a connection with a client device; receiving a request from said client device to upload a digital object to said object storage system, the complete contents of said digital object to be uploaded being unknown to said client device at the time said request is made; receiving a series of chunks of said digital object from by said client device responsive to said request; receiving a trailer from said client device following a final chunk of said series of chunks, said trailer including at least one parameter associated with said complete contents of said digital object uploaded by said client device; and verifying that said complete contents of said digital object has been received using said at least one parameter included in said trailer.

2 . The method of claim 1 , wherein said request from said client device comprises an HTTP request specifying transfer-encoding chunked.

3 . The method of claim 1 , wherein said at least one parameter comprises a checksum of said series of chunks uploaded by said client device.

4 . The method of claim 3 , further comprising:
assembling said series of chunks into an assembled digital object; and determining a checksum of said assembled digital object; and wherein said step of verifying that said complete contents of said digital object has been received includes comparing said checksum of said assembled digital object to said checksum included in said trailer.

5 . The method of claim 1 , further comprising:
assembling said series of chunks into an assembled digital object; storing said assembled digital object; and encrypting said assembled digital object using an encryption key unique to said assembled digital object.

6 . The method of claim 1 , wherein said at least one parameter comprises a size of said series of chunks uploaded by said client device.

7 . An object storage system for storing digital objects, said object storage system comprising:
at least one storage node including memory for storing digital objects therein; a client interface facilitating a connection with a client device; and an upload service operative to
receive a request from said client device to upload a digital object to said object storage system, the complete contents of said digital object to be uploaded being unknown to said client device at the time said request is made,
receive a series of chunks of said digital object from said client device responsive to said request,
receive a trailer from said client device following a final chunk of said series of chunks, said trailer including at least one parameter associated with said complete contents of said digital object uploaded by said client device, and
verify that said complete contents of said digital object has been received using said at least one parameter included in said trailer.

8 . The object storage system of claim 7 , wherein said request comprises an HTTP request specifying transfer-encoding chunked.

9 . The object storage system of claim 7 , wherein said at least one parameter comprises a checksum of said series of chunks uploaded by said client device.

10 . The object storage system of claim 9 , wherein said upload service is further operative to:
assemble said series of chunks into an assembled digital object; determine a checksum of said assembled digital object; and compare said checksum of said assembled digital object to said checksum included in said trailer to verify that said complete contents of said digital object has been received by said cloud object store.

11 . The object storage system of claim 7 , wherein said upload service is further operative to:
assemble said series of chunks into an assembled digital object; store said assembled digital object; and encrypt said assembled digital object using an encryption key unique to said assembled digital object.

12 . The object storage system of claim 7 , wherein said at least one parameter comprises a size of said series of chunks uploaded by said client device.

13 . In a client device, a method for uploading a streamed object to an object storage system, said method comprising:
receiving a streamed object from a stream source; establishing a connection with said object storage system; providing a request to upload said streamed object as a series of chunks to said object storage system, the complete contents of said streamed object being unknown to said client device at the time of said request; preparing said series of chunks associated with said streamed object consistent with said request; uploading each chunk in said series of chunks to said object storage system; computing at least one parameter indicative of the complete contents of said series of chunks uploaded to said object storage system; and providing a trailer to said object storage system following a final chunk of said series, said trailer including said at least one parameter.

14 . The method of claim 13 , wherein said request comprises an HTTP request specifying transfer-encoding chunked.

15 . The method of claim 13 , wherein said step of computing said at least one parameter comprises computing a checksum of said series of chunks uploaded to said object storage system.

16 . The method of claim 15 , wherein said trailer includes said checksum of said series of chunks uploaded to said object storage system.

17 . The method of claim 13 , further comprising processing said streamed object according to a client-specific process prior to uploading each said chunk in said series.

18 . A client device configured to upload a streamed object to an object storage system, said client device comprising:
a streamed object interface configured to receive a streamed object from a stream source; an object store interface facilitating a connection with an object storage system; and a stream upload application operative to
provide a request to upload said streamed object to said object storage system as a series of chunks via said object store interface, the complete contents of said streamed object being unknown to said client device at the time of said request,
prepare said series of chunks associated with said streamed object consistent with said request,
upload each chunk in said series of chunks to said object storage system via said object store interface,
compute at least one parameter indicative of the complete contents of said series of chunks uploaded to said object storage system, and
provide a trailer to said object storage system via said object store interface following a final chunk of said series, said trailer including said at least one parameter.

19 . The client device of claim 18 , wherein said request comprises an HTTP request specifying transfer-encoding chunked.

20 . The client device of claim 18 , wherein said at least one parameter comprises a checksum of said series of chunks uploaded to said object storage system.

21 . The client device of claim 20 , wherein said trailer includes said checksum of said series of chunks uploaded to said object storage system.

22 . The client device of claim 18 , wherein said stream upload application is further operative to process said streamed object according to a client-specific process prior to uploading each said chunk in said series of chunks.

Assignees

• Egnyte Inc

Inventors

• Shetty Sachin
• Jassal Amrit
• Sharma Shishir
• Rai Yogesh
• Chauhan Manoj
• Jakubowski Leszek
• Lahiri Krishanu

Classifications

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F16/182

  Distributed file systems · CPC title

• H04L63/0209

  Architectural arrangements, e.g. perimeter networks or demilitarized zones · CPC title

• H04L63/0281 Primary

  Proxies · CPC title

• H04L63/0272

  Virtual private networks · CPC title