Inserting executions of owner-specified code into input/output path of object storage service

Patent metadata

Field	Value
Publication number	US-11055112-B2
Application number	US-201916586704-A
Country	US
Kind code	B2
Filing date	Sep 27, 2019
Priority date	Sep 27, 2019
Publication date	Jul 6, 2021
Grant date	Jul 6, 2021

Abstract

Systems and methods are described for modifying input and output (I/O) to an object storage service by implementing one or more owner-specified functions to I/O requests. A function can implement a data manipulation, such as filtering out sensitive data before reading or writing the data. The functions can be applied prior to implementing a request method (e.g., GET or PUT) specified within the I/O request, such that the data to which the method is applied my not match the object specified within the request. For example, a user may request to obtain (e.g., GET) a data set. The data set may be passed to a function that filters sensitive data to the data set, and the GET request method may then be applied to the output of the function. In this manner, owners of objects on an object storage service are provided with greater control of objects stored or retrieved from the service.

First claim

What is claimed is:

1. A system to enable application of data manipulations to input/output (IO) operations requesting retrieval of data objects stored on an object storage service, the system comprising:
one or more computing devices implementing the object storage service, the service storing a collection of data objects on behalf of an owner; and
one or more processors configured with computer-executable instructions to:
receive a request from a computing device of the owner to insert execution of owner-defined code into an input/output (IO) path for the collection of data objects;
store an association of the IO path with the owner-defined code that configures the object storage service to, prior to satisfying requests received via the IO path for the collection of data objects, initiate an execution of the owner-defined code; and

wherein the object storage service is configured by the stored association of the IO path with the owner-defined code to:
receive, via the IO path, an IO request associated with the collection of data objects, wherein the IO request specifies a request method to be applied to input data; and
satisfy the IO request at least partly by initiating the execution of the owner-defined code against the input data to produce output data, and applying the request method to the output data.

2. The system of claim 1 , wherein the request method is hypertext transport protocol (HTTP) GET, wherein the input data is an object within the collection of objects to be retrieved by the request, and wherein applying the request method to the output data comprises returning the output data as the object.

3. The system of claim 1 , wherein the IO path is defined at least in part based on one or more of the collection of data objects, a resource identifier to which the IO request is transmitted, or a computing device from which the IO request is received.

4. The system of claim 1 , wherein the one or more processors are further configured with the computer-executable instructions to:
receive a request from the computing device of the owner to insert execution of second owner-defined code into a second input/output (IO) path for the collection of data objects;
store an association of the second IO path with the second owner-defined code that configures the object storage service to, prior to satisfying requests received via the second IO path for to the collection of data objects, execute the second owner-defined code; and
wherein the object storage service is configured by the stored association of the second IO path with the second owner-defined code to:
receive, via the second IO path, a second IO request associated with the collection of data objects, wherein the second IO request specifies an additional request method to be applied to second input data; and
satisfy the second IO request at least partly by initiating an execution of the second owner-defined code against the second input data to produce second output data, and applying the additional request method to the second output data.

5. A computer-implemented method comprising:
receiving a request from a computing device of an owner to insert execution of owner-defined code into an input/output (IO) path for a collection of data objects stored on an object storage service on behalf of the owner; and
storing an association of the IO path with the owner-defined code that configures the object storage service to, prior to satisfying IO requests for IO to the collection of data objects, execute the owner-defined code;
wherein configuring the object storage service causes the object storage service, in response to an IO request associated with the collection of data objects and specifying a request method to be applied to input data, to satisfy the IO request at least partly by:
initiating an execution of the owner-defined code against the input data to produce output data, and
applying the request method to the output data.

6. The computer-implemented method of claim 5 , wherein the owner-defined code is provided by the computing device of the owner.

7. The computer-implemented method of claim 5 , wherein the owner-defined code is selected from a library of code associated with the object storage service.

8. The computer-implemented method of claim 5 , wherein the request method is hypertext transport protocol (HTTP) PUT, wherein the input data is provided within the IO request, and wherein applying the request method to the output data comprises storing the output data as a new object within the collection of objects.

9. The computer-implemented method of claim 5 , wherein the IO request includes a call to second owner-defined code associated with the collection of data objects, and wherein the object storage service is configured, in response to the IO request, to execute the second owner-defined code to produce the input data.

10. The computer-implemented method of claim 5 , wherein IO request does not reference the owner-defined code.

11. The computer-implemented method of claim 5 , wherein to satisfy the IO request at least partly by initiating the execution of the owner-defined code against the input data to produce output data, the object storage service is further configured to pass to the execution a handle providing access to the input data as an IO stream.

12. The computer-implemented method of claim 11 , wherein to satisfy the IO request at least partly by initiating the execution of the owner-defined code against the input data to produce output data, the object storage service is further configured to stage a copy of the input data on a local file system of the execution.

13. Non-transitory computer-readable media storing computer-executable instructions that, when executed by a data object storage system, cause the data object storage system to:
receive a request from a computing device of an owner to insert execution of owner-defined code into an input/output (IO) path for a collection of data objects stored on an object storage service on behalf of the owner;
store an association of the IO path with the owner-defined code;
receive an IO request associated with the collection of data objects, wherein the IO request specifies a request method to be applied to input data; and
responsive to the stored association of the IO path with the owner-defined code, satisfy the IO request at least partly by initiating an execution of the owner-defined code against the input data to produce output data, and applying the request method to the output data.

14. The non-transitory computer-readable media of claim 13 , wherein the request method is hypertext transport protocol (HTTP) LIST, wherein the input data is a manifest of data objects within the collection of data objects, and wherein applying the request method to the output data comprises returning the output data as the manifest of data objects within the collection of data objects.

15. The non-transitory computer-readable media of claim 14 , wherein the output data excludes identification of at least one data object within the collection of data objects removed from the manifest during execution of the owner-defined code.

16. The non-transitory computer-readable media of claim 13 , wherein to satisfy the IO request at least partly by initiating the execution of the owner-defined code against the input data to produce output data, the object storage service is further configured to pass to the execution a file handle providing access to the input data as an IO stream.

17. The non-transitory computer-readable media of claim 13 , wherein to

Assignees

• Amazon Tech Inc

Inventors

• Miller Kevin C
• Datta Ramyanshu
• Harris Timothy Lawrence

Classifications

• G06F2209/5011

  Pool · CPC title

• G06F13/1668 Primary

  Details of memory controller · CPC title

• G06F9/44568

  Immediately runnable code · CPC title

• G06F9/547

  Remote procedure calls [RPC]; Web services · CPC title

• G06F9/44573 Primary

  Execute-in-place [XIP] · CPC title