Method and device for data replication

Patent metadata

Field	Value
Publication number	US-10459642-B2
Application number	US-201615387841-A
Country	US
Kind code	B2
Filing date	Dec 22, 2016
Priority date	Dec 30, 2015
Publication date	Oct 29, 2019
Grant date	Oct 29, 2019

Abstract

There is provided a method and device for data replication. The method comprises: obtaining, in a network interface card, data segments by segmenting input first data; determining, in the network interface card, fingerprints corresponding to the data segments; and comparing, in a central processing unit, the fingerprints of the data segments with existing fingerprints corresponding to processed data segments, and determining, based on a result of the comparing, whether to de-duplicate the data segments corresponding to the fingerprints, to perform the data replication.

First claim

What is claimed is:

1. A method for data replication, comprising:
obtaining, in a network interface card, data segments by segmenting input first data;
determining, in the network interface card, fingerprints corresponding to the data segments; and
comparing, in a central processing unit, the fingerprints of the data segments with existing fingerprints corresponding to processed data segments, and determining, based on a result of the comparing, whether to de-duplicate the data segments corresponding to the fingerprints, to perform the data replication;
wherein the network interface card and the CPU reside within a data replication hardware appliance operating as replication destination equipment;
wherein the data segments are received from replication source equipment that is separated from the replication destination equipment by a network; and
wherein the method further comprises:
prior to obtaining the data segments, establishing a replication session between the replication source equipment and the data replication hardware appliance, the replication session routinely replicating data from the replication source equipment to the data replication hardware appliance through the network.

2. The method according to claim 1 , wherein the network interface card is configured as a field programmable gate array (FPGA)-based network interface card.

3. The method according to claim 1 , wherein determining, in the network interface card, fingerprints corresponding to the data segments comprises:
determining, in the network interface card, the fingerprints corresponding to the data segments based on a secure hash algorithm.

4. The method according to claim 1 , further comprising:
prior to segmenting the first data, obtaining, through protocol processing, the first data from a data packet entering into the network interface card.

5. The method according to claim 1 , wherein comparing, in a central processing unit, the fingerprints of the data segments with existing fingerprints corresponding to processed data segments, and determining, based on a result of the comparing, whether to de-duplicate the data segments corresponding to the fingerprints comprises:
if a fingerprint matches with any of the existing fingerprints, de-duplicating, by the central processing unit, the data segment corresponding to the fingerprint.

6. The method according to claim 1 , wherein comparing, in a central processing unit, the fingerprints of the data segments with existing fingerprints corresponding to processed data segments, and determining, based on a result of the comparing, whether to de-duplicate the data segments corresponding to the fingerprints comprises:
if a fingerprint matches with none of the existing fingerprints, determining, by the central processing unit, that the data segment corresponding to the fingerprint is a data segment unable to be de-duplicated; and
compressing, in the network interface card, the data segment unable to be de-duplicated.

7. The method according to claim 6 , further comprising:
encrypting the compressed data segment in the network interface card.

8. The method according to claim 1 , further comprising:
after determining the fingerprints corresponding to the data segments, writing the fingerprints together with the data segments corresponding to the fingerprints from the network interface card to a first cache.

9. The method according to claim 6 , further comprising:
prior to comparing the fingerprints with the existing fingerprints, reading, by the central processing unit, the fingerprints from a first cache.

10. The method according to claim 6 , further comprising:
after comparing the fingerprints with the existing fingerprints, reading, by the central processing unit, from a first cache the data segment unable to be de-duplicated for used in compressing in the network interface card.

11. The method according to claim 6 , further comprising:
after the compressing or the encrypting, writing into a second cache the data segment compressed or encrypted in the network interface card.

12. The method according to claim 1 , wherein the data replication hardware appliance includes a set of storage devices; and
wherein the method further comprises:
receiving, by the network interface card residing within the data replication hardware appliance, a replication communication from the replication source equipment, the replication communication including a first data segment and a second data segment,
generating, by the network interface card residing within the data replication hardware appliance, a first fingerprint based on the first data segment and a second fingerprint based on the second data segment,
performing, by the central processing unit residing within the data replication hardware appliance, a first match operation that successfully matches the first fingerprint to a table of existing fingerprints and a second match operation that unsuccessfully matches the second fingerprint to the table of existing fingerprints,
in response to a first result of the first match operation, de-duplicating the first data segment to alleviating writing the first data segment to the set of storage devices,
in response to a second result of the second match operation, writing the second data segment to the set of storage devices, and
after de-duplicating the first data segment and writing the second data segment, providing a reply from the data replication hardware appliance to the replication source equipment, the reply fully acknowledging data replication in response to the replication communication that includes the first data segment and the second data segment.

13. A device for data replication, comprising:
a network interface card configured to:
obtain data segments by segmenting input first data, and
determine fingerprints corresponding to the data segments; and

a central processing unit configured to:
compare the fingerprints of the data segments with existing fingerprints corresponding to processed data segments, and
determine, based on a result of the comparing, whether to de-duplicate the data segments corresponding to the fingerprints, to perform the data replication;

wherein the device for data replication is a data replication hardware appliance operating as replication destination equipment;
wherein the data segments are received from replication source equipment that is separated from the replication destination equipment by a network; and
wherein the data replication hardware appliance is constructed and arranged to:
prior to obtaining the data segments, establish a replication session between the replication source equipment and the data replication hardware appliance, the replication session routinely replicating data from the replication source equipment to the data replication hardware appliance through the network.

14. The device according to claim 13 , wherein the network interface card is configured as a field programmable gate array (FPGA)-based network interface card.

15. The device according to claim 13 , wherein the network interface card is further configured to:
determine the fingerprints corresponding to the data segments based on a secure hash algorithm.

16. The device according to claim 13 , wherein the network interface card is further configured to:
prior to segmenting the first data, obtain, through protocol processing, the first data from a data packet entering into the network interface card.

17. The device according to claim 13 , wherein the central processing unit is further configured to

Assignees

• Emc Ip Holding Co Llc

Inventors

• Wang Kun
• Zou Colin Yong
• Ye Sean Cheng
• Li Lyne Yuwei

Classifications

• H04L63/123

  received data contents, e.g. message integrity · CPC title

• H04L63/0428

  wherein the data content is protected, e.g. by encrypting or encapsulating the payload · CPC title

• H04L67/1097 Primary

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F3/065

  Replication mechanisms · CPC title

• G06F3/0619 Primary

  in relation to data integrity, e.g. data losses, bit errors · CPC title