File operation task optimization

Patent metadata

Field	Value
Publication number	US-12547455-B2
Application number	US-202318317081-A
Country	US
Kind code	B2
Filing date	May 13, 2023
Priority date	Nov 25, 2013
Publication date	Feb 10, 2026
Grant date	Feb 10, 2026

Abstract

A method includes receiving, by a data processing apparatus, a plurality of file operation requests, each file operation request including a priority, a deadline, and an operation type and representing a request to perform an operation on at least one file maintained in a distributed file system; identifying, by the data processing apparatus, a group of file operation requests to be executed together from the plurality of file operation requests, the identification based at least in part on at least one of: the file operations in the group of file operations being directed to a same storage system, or file operations in the group of file operations sharing a common operation type; and sending a request to execute the group of file operation requests to a system configured to perform the group of file operation requests.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
receiving, by data processing hardware and from a first requestor, a first operation request specifying a first operation to perform on a first file; receiving, by the data processing hardware and from a second requestor different from the first requestor, a second operation request specifying a second operation to perform on a second file, the second operation request different from the first operation request and the second file different from the first file; determining, by the data processing hardware, that each of the first operation request and the second operation request are directed to a distributed file system storing the first file and the second file; based on determining that each of the first operation request and the second operation request are directed to the distributed file system, dynamic grouping, by the data processing hardware, of the first operation request and the second operation request as received into a single request, wherein the dynamic grouping groups operation requests as the operation requests are received based on at least criteria for grouping including one or more of target storage system, operation type, deadline, or priority; and sending, by the data processing hardware, the single request to the distributed file system to request performance of:
the first operation on the first file as specified by the first operation request; and
the second operation on the second file as specified by the second operation request.

2 . The method of claim 1 , wherein the first operation and the second operation each comprise one of:
a file copy operation; a file deletion operation; or a file merge operation.

3 . The method of claim 2 , wherein the first operation and the second operation are the same type.

4 . The method of claim 2 , wherein the first operation and the second operation are different types.

5 . The method of claim 1 , wherein each of the first operation request and the second operation request comprise a respective priority indicating an urgency of the respective operation request.

6 . The method of claim 1 , wherein each of the first operation request and the second operation request comprise a respective process association indicating a particular process of the respective operation request.

7 . The method of claim 1 , wherein sending the single request comprises sending a remote procedure call.

8 . The method of claim 1 , wherein:
the first operation request comprises a first soft deadline and a first hard deadline for executing the first operation specified by the first operation request; and the second operation request comprises a second soft deadline and a second hard deadline for executing the second operation specified by the second operation request, wherein a soft deadline requests that an operation complete before a deadline, and a hard deadline requires the operation to complete before a deadline or the operation be aborted.

9 . The method of claim 8 , further comprising:
determining, by the data processing hardware, that the single request will not be completed before one of:
the first hard deadline; or
the second hard deadline; and
aborting, by the data processing hardware, the single request.

10 . The method of claim 1 , further comprising:
receiving, by the data processing hardware, a third operation request specifying a third operation to perform on a third file; determining, by the data processing hardware, that the third file is not stored on the distributed file system storing the first file and the second file; and based on determining that the third file is not stored on the distributed file system storing the first file and the second file, excluding, by the data processing hardware, the third operation request from the single request.

11 . A system comprising:
data processing hardware; and memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed by the data processing hardware cause the data processing hardware to:
receive, from a first requestor, a first operation request specifying a first operation to perform on a first file;
receive, from a second requestor different from the first requestor, a second operation request specifying a second operation to perform on a second file, the second operation request different from the first operation request and the second file different from the first file;
determine that each of the first operation request and the second operation request are to be issued to a distributed file system storing the first file and the second file;
based on a determination that each of the first operation request and the second operation request are to be issued to the distributed file system, dynamically group the first operation request and the second operation request into a single request as operation requests are received based on at least criteria including one or more of target storage system, operation type, deadline, or priority; and
send the single request to the distributed file system to request performance of:
the first operation on the first file as specified by the first operation request; and
the second operation on the second file as specified by the second operation request.

12 . The system of claim 11 , wherein the first operation and the second operation each comprise one of:
a file copy operation; a file deletion operation; or a file merge operation.

13 . The system of claim 12 , wherein the first operation and the second operation are the same type.

14 . The system of claim 12 , wherein the first operation and the second operation are different types.

15 . The system of claim 11 , wherein each of the first operation request and the second operation request comprise a respective priority indicating an urgency of the respective operation request.

16 . The system of claim 11 , wherein each of the first operation request and the second operation request comprise a respective process association indicating a particular process of the respective operation request.

17 . The system of claim 11 , wherein, to send the single request, the instructions cause the data processing hardware to send a remote procedure call.

18 . The system of claim 11 , wherein:
the first operation request comprises a first soft deadline and a first hard deadline for executing the first operation specified by the first operation request; and the second operation request comprises a second soft deadline and a second hard deadline for executing the second operation specified by the second operation request.

19 . The system of claim 18 , wherein the instructions further cause the data processing hardware to:
determine that the single request will not be completed before one of:
the first hard deadline; or
the second hard deadline; and
abort the single request.

20 . The system of claim 11 , wherein the instructions further cause the data processing hardware to:
receive a third operation request specifying a third operation to perform on a third file; determine that the third file is not stored on the distributed file system storing the first file and the second file; and based on determining that the third file is not stored on the distributed file system storing the first file and the second file, exclude the third operation request from the single request.

Assignees

• Google Llc

Inventors

• Ma Chi
• Goldman Kenneth J
• Zhao Yonggang
• Gildea Stephen P G

Classifications

• G06F9/4887

  involving deadlines, e.g. rate based, periodic · CPC title

• G06F16/182 Primary

  Distributed file systems · CPC title

• G06F16/16

  File or folder operations, e.g. details of user interfaces specifically adapted to file systems · CPC title

• G06F9/4881 Primary

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title