Data deduplication system

Patent metadata

Field	Value
Publication number	US-11429573-B2
Application number	US-201916654995-A
Country	US
Kind code	B2
Filing date	Oct 16, 2019
Priority date	Oct 16, 2019
Publication date	Aug 30, 2022
Grant date	Aug 30, 2022

Abstract

A data deduplication system includes a data deduplication subsystem coupled to each of a host system and a storage system. The data deduplication system receives data from the host system, generates a data deduplication identifier for the data, and determines whether the data deduplication identifier for the data is stored in a data deduplication database. In response to determining that the data deduplication identifier is not stored in the data deduplication database, the data deduplication system stores the data deduplication identifier for the data in the data deduplication database in association with a data counter for the data, and transmits the data to the storage system for storage. In response to determining that the data deduplication identifier is stored in the data deduplication database, the data deduplication system increments a data counter that is associated with the data deduplication identifier in the data deduplication database, and discards the data.

First claim

What is claimed is:

1. A data deduplication system, comprising:
a host system that is provided in a host system chassis;
a storage system that is provided in a storage system chassis that is separate from the host system chassis;
a Software Defined Networking (SDN) controller system that includes a first portion of a data deduplication database; and
a first network switch device that is provided in a first network switch device chassis that is separate from the host system chassis and the storage system chassis, and that is coupled to each of the host system, the storage system, and the SDN controller system via a first network switch device, wherein the first network switch device includes a second portion of the data deduplication database and provides a first data deduplication subsystem that is configured to:
receive, via a network using the first network switch device, first data transmitted by the host system through the network for storage in the storage system;
generate a first data deduplication identifier for the first data;
determine whether the first data deduplication identifier for the first data is stored in either of the first portion of the data deduplication database or the second portion of the data deduplication database;
in response to determining that the first data deduplication identifier for the first data is not stored in either of the first portion of the data deduplication database or the second portion of the data deduplication database:
store the first data deduplication identifier for the first data in the second portion of the data deduplication database in association with a first data counter for the first data; and
transmit, through the network using the first network switch device, the first data to the storage system for storage; and

in response to determining that the first data deduplication identifier for the first data is stored in one of the first portion of the data deduplication database and the second portion of the data deduplication database:
increment a first data counter that is associated with the first data deduplication identifier for the first data in the one of the first portion of the data deduplication database and the second portion of the data deduplication database; and
discard the first data.

2. The system of claim 1 , further comprising:
a second network switch device that is provided in a second network switch device chassis that is separate from the host system chassis and the storage system chassis, and that is coupled to each of the host system, the storage system, and the SDN controller system via a second network switch device, wherein the second network switch device includes a third portion of the data deduplication database and provides a second data deduplication subsystem that is configured to:
receive, via the network using the second network switch device, second data transmitted by the host system through the network for storage in the storage system;
generate a second data deduplication identifier for the second data;
determine whether the second data deduplication identifier for the second data is stored in either of the first portion of the data deduplication database or the third portion of the data deduplication database;
in response to determining that the second data deduplication identifier for the second data is not stored in either of the first portion of the data deduplication database or the third portion of the data deduplication database:
store the second data deduplication identifier for the second data in the third portion of the data deduplication database in association with a second data counter for the second data; and
transmit, through the network using the second network switch device, the second data to the storage system for storage; and

in response to determining that the second data deduplication identifier for the second data is stored in one of the first portion of the data deduplication database and the third portion of the data deduplication database:
increment a second data counter that is associated with the second data deduplication identifier for the second data in the one of the first portion of the data deduplication database and the third portion of the data deduplication database; and
discard the second data.

3. The system of claim 2 , wherein the second data deduplication system is configured to:
transfer the second data deduplication identifier for the second data and the second data counter for the second data from the third portion of the data deduplication database to the first portion of the data deduplication database.

4. The system of claim 1 , wherein the SDN controller system is provided in a SDN controller system chassis that is separate from the host system chassis, the storage system chassis, and the first network switch device chassis.

5. The system of claim 1 , wherein the first data deduplication subsystem is configured to:
transfer the first data deduplication identifier for the first data and the first data counter for the first data from the second portion of the data deduplication database to the first portion of the data deduplication database.

6. The system of claim 1 , wherein the first data deduplication subsystem is configured to:
receive a data delete instruction from the host system that identifies the first data for deletion;
decrement the first data counter that is associated with the first data deduplication identifier for the first data in the one of the first portion of the data deduplication database and the second portion of the data deduplication database;
determine whether the first data counter that is associated with the first data deduplication identifier for the first data in the one of the first portion of the data deduplication database and the second portion of the data deduplication database is at zero; and
cause, in response to determining that the first data counter that is associated with the first data deduplication identifier for the first data in the one of the first portion of the data deduplication database and the second portion of the data deduplication database is at zero, the first data to be deleted from the storage system.

7. An Information Handling System (IHS), comprising:
at least one first Information Handling System (IHS) chassis;
a first processing system that is housed in the at least one first IHS chassis; and
a first memory system that is housed in the at least one first IHS chassis, coupled to the first processing system, and that includes instructions that, when executed by the first processing system, cause the first processing system to provide a first data deduplication engine that is configured to:
receive, via a network using a first network switch device and from a host system that is provided in a host system chassis that is separate from the at least one first IHS chassis, first data transmitted by the host system through the network for storage in a storage system that is provided in a storage system chassis that is separate from the at least one first IHS chassis and the host system chassis;
generate a first data deduplication identifier for the first data;
determine whether the first data deduplication identifier for the first data is stored in either of a first portion of a data deduplication database that is included in a Software Defined Networking (SDN) controller system or a second portion of the data deduplication database that is included in the first network switch device;
in response to determining that the first data deduplication identifier for the first data is not stored in one of the first portion of the data deduplication database and the second portion of the data deduplication database:
store the first data deduplication iden

Assignees

• Dell Products Lp

Inventors

• Patel Dharmesh M
• Chaganti Ravikanth
• Ali Rizwan

Classifications

• G06F16/215 Primary

  Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors · CPC title

• G06F16/1748 Primary

  De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641) · CPC title