Methods and systems for dynamically adjusting data chunk sizes copied over a network

Patent metadata

Field	Value
Publication number	US-2023344902-A1
Application number	US-202217724660-A
Country	US
Kind code	A1
Filing date	Apr 20, 2022
Priority date	Apr 20, 2022
Publication date	Oct 26, 2023
Grant date	—

Abstract

A method for copying source data from a source server to a destination server, that includes initiating, by a source copy manager, a copy operation of the source data, where during the copy operation, the method further includes obtaining resource statistics, setting a data chunk size based on the resource statistics, copying a data chunk to a read queue, where the data chunk includes a portion of the source data, and where the data chunk has the data chunk size, and sending, from the read queue, the data chunk to a network device, and ending the copy operation.

First claim

1 . A method for copying source data from a source server to a destination server, comprising:
initiating, by a source copy manager, a copy operation of the source data, wherein during the copy operation, the method further comprises:
obtaining resource statistics at a first time to obtain first resource statistics, wherein the resource statistics comprises at least one of the following: processing usage of the source server, processing usage of the source server due to the copy operation, processing usage of the source server due to operations other than the copy operation, memory usage of the source server, memory usage of the source server due to the copy operation, and memory usage of the source server due to operations other than the copy operation;
setting a first data chunk size based on the first resource statistics;
copying a first data chunk to a read queue, wherein the first data chunk comprises a portion of the source data, and wherein the first data chunk has the first data chunk size;
sending, from the read queue, the first data chunk to a network device; and
after sending:
obtaining the resource statistics at a second time to obtain second resource statistics;
setting a second data chunk size different than the first data chunk size and based on the second resource statistics;
copying a second data chunk to the read queue, wherein the second data chunk comprises a second portion of the source data, and wherein the second data chunk has the second data chunk size; and
sending, from the read queue, the second data chunk to the network device.

2 . The method of claim 1 , wherein setting the first data chunk size based on the first resource statistics, comprises:
analyzing the first resource statistics; making a first determination, based on the first resource statistics, that resources of the source server are underutilized; and in response to the first determination:
setting the first data chunk size to be larger than a previous data chunk size.

3 . (canceled)

4 . The method of claim 1 , wherein setting the second data chunk size based on the second resource statistics, comprises:
analyzing the second resource statistics; making a second determination, based on the second resource statistics, that the resources of the source server are underutilized; and in response to the second determination:
setting the second data chunk size to be larger than the first data chunk size.

5 . The method of claim 1 , wherein setting the first data chunk size based on the first resource statistics, comprises:
analyzing the first resource statistics; making a first determination, based on the first resource statistics, that a resource of the source server is utilized above a threshold; and in response to the first determination:
setting the first data chunk size to be smaller than a previous data chunk size.

6 . The method of claim 5 , wherein the resource of the source server is being utilized by a concurrent user operation executing on the source server.

7 . The method of claim 1 , wherein prior to setting the first data chunk size, the source server receives a retry request, and wherein setting the first data chunk size comprises setting the first data chunk size to be smaller than a previous data chunk size.

8 . A non-transitory computer readable medium comprising instructions which, when executed by a computer processor, enables the computer processor to perform a method for copying source data from a source server to a destination server, comprising:
initiating, by a source copy manager, a copy operation of the source data, wherein during the copy operation, the method further comprises:
obtaining resource statistics at a first time to obtain first resource statistics, wherein the resource statistics comprises at least one of the following: processing usage of the source server, processing usage of the source server due to the copy operation, processing usage of the source server due to operations other than the copy operation, memory usage of the source server, memory usage of the source server due to the copy operation, and memory usage of the source server due to operations other than the copy operation;
setting a first data chunk size based on the first resource statistics;
copying a first data chunk to a read queue, wherein the first data chunk comprises a portion of the source data, and wherein the first data chunk has the first data chunk size;
sending, from the read queue, the first data chunk to a network device; and
after sending:
obtaining the resource statistics at a second time to obtain second resource statistics;
setting a second data chunk size different than the first data chunk size and based on the second resource statistics;
copying a second data chunk to the read queue, wherein the second data chunk comprises a second portion of the source data, and wherein the second data chunk has the second data chunk size; and
sending, from the read queue, the second data chunk to the network device.

9 . The non-transitory computer readable medium of claim 8 , wherein setting the first data chunk size based on the first resource statistics, comprises:
analyzing the first resource statistics; making a first determination, based on the first resource statistics, that resources of the source server are underutilized; and in response to the first determination:
setting the first data chunk size to be larger than a previous data chunk size.

10 . (canceled)

11 . The non-transitory computer readable medium of claim 8 , wherein setting the second data chunk size based on the second resource statistics, comprises:
analyzing the second resource statistics; making a second determination, based on the second resource statistics, that the resources of the source server are underutilized; and in response to the second determination:
setting the second data chunk size to be larger than the first data chunk size.

12 . The non-transitory computer readable medium of claim 8 , wherein setting the first data chunk size based on the first resource statistics, comprises:
analyzing the first resource statistics; making a first determination, based on the first resource statistics, that a resource of the source server is utilized above a threshold; and in response to the first determination:
setting the first data chunk size to be smaller than a previous data chunk size.

13 . The non-transitory computer readable medium of claim 12 , wherein the resource of the source server is being utilized by a concurrent user operation executing on the source server.

14 . The non-transitory computer readable medium of claim 8 , wherein prior to setting the first data chunk size, the source server receives a retry request, and wherein setting the first data chunk size comprises setting the first data chunk size to be smaller than a previous data chunk size.

15 . A source server, comprising:
memory comprising a read queue; a storage device comprising source data; and a processor executing a source copy manager, wherein the processor is configured to perform a method for copying the source data from to a destination server, comprising: initiating, by the source copy manager, a copy operation of the source data, wherein during the copy operation, the method further comprises:
obtaining resource statistics at a first time to obtain first resource statistics, wherein the resource statistics comprises at least one of the following: processor usage of the source server, processor usage of the source server due to the copy operation, processor usage of the source

Assignees

• Dell Products Lp

Inventors

• Yadav Sunil
• Chopra Shelesh
• Bansal Aaditya
• Sharma Manish

Classifications

• H04L67/1095 Primary

  Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes · CPC title

• H04L47/628

  based on packet size, e.g. shortest packet first · CPC title

• H04L47/765 Primary

  triggered by the end-points · CPC title