Application aware export to object storage of low-reference data in deduplication repositories

Patent metadata

Field	Value
Publication number	US-10956273-B2
Application number	US-201615082222-A
Country	US
Kind code	B2
Filing date	Mar 28, 2016
Priority date	Mar 28, 2016
Publication date	Mar 23, 2021
Grant date	Mar 23, 2021

Abstract

Various embodiments for managing data in a data deduplication repository in a computing storage environment, by a processor device, are provided. In one embodiment, a method comprises migrating certain deduplication repository data from a host to an object storage by integrating a data offload process with an existing backup management application, for reducing unnecessary deduplication repository data stored on the host.

First claim

What is claimed is:

1. A method for managing data in a data deduplication repository in a computing storage environment, by a processor device, comprising:
migrating certain deduplication repository data from a host to an object storage by integrating a data offload process with an existing backup management application executing on the host, for reducing unnecessary deduplication repository data stored on the host; wherein the certain deduplication repository data comprises data identified by the existing backup management application as candidate repository data developed on a candidate list; and the candidate repository data on the candidate list includes repository data having a reference count number below a predetermined reference count threshold, the reference count number associated with a deduplication ratio;
issuing a first application programming interface (API) command, by the existing backup management application to a deduplication gateway having input specifying to locate any data within a subset of entities having the reference count number below the predetermined reference count threshold and a data age older than a predetermined age threshold; wherein upon receiving the first API command, the deduplication gateway scans metadata of the subset of entities in a deduplication repository to identify the candidate repository data according to the first API command input, the subset of entities comprising at least one of a range of virtual cartridges, all cartridges in a virtual library, and all cartridges in the deduplication repository; and
responsive to receiving the candidate list by the backup management application, initiating the data offload process by migrating the candidate repository data on the candidate list from the host to the object storage using a second API command by the existing backup management application; wherein, prior to migrating the candidate repository data, upon determining that a repository storing the certain deduplication repository data on the host has exceeded a predetermined repository capacity threshold, user data and metadata associated with virtual entities identified in the subset of virtual entities comprising the candidate repository data is converted, using a representation state transfer (REST) API, into objects and containers comprised of data structures storing the objects subsequently transferred from the host to the object storage upon receipt of the second API command.

2. The method of claim 1 , further including excluding repository data from the candidate list based on at least one of the predetermined age threshold and repository data marked as excluded by an end user.

3. The method of claim 1 , further including maintaining a mapping of the migrated certain deduplication repository data between the host and the object storage by updating host repository metadata.

4. A system for managing data in a data deduplication repository in a computing storage environment, the system comprising:
at least one processor device, wherein the at least one processor device:
migrates certain deduplication repository data from a host to an object storage by integrating a data offload process with an existing backup management application executing on the host, for reducing unnecessary deduplication repository data stored on the host; wherein the certain deduplication repository data comprises data identified by the existing backup management application as candidate repository data developed on a candidate list; and the candidate repository data on the candidate list includes repository data having a reference count number below a predetermined reference count threshold, the reference count number associated with a deduplication ratio;

issues a first application programming interface (API) command, by the existing backup management application to a deduplication gateway having input specifying to locate any data within a subset of entities having the reference count number below the predetermined reference count threshold and a data age older than a predetermined age threshold; wherein upon receiving the first API command, the deduplication gateway scans metadata of the subset of entities in a deduplication repository to identify the candidate repository data according to the first API command input, the subset of entities comprising at least one of a range of virtual cartridges, all cartridges in a virtual library, and all cartridges in the deduplication repository; and
responsive to receiving the candidate list by the backup management application, initiates the data offload process by migrating the candidate repository data on the candidate list from the host to the object storage using a second API command by the existing backup management application; wherein, prior to migrating the candidate repository data, upon determining that a repository storing the certain deduplication repository data on the host has exceeded a predetermined repository capacity threshold, user data and metadata associated with virtual entities identified in the subset of virtual entities comprising the candidate repository data is converted, using a representation state transfer (REST) API, into objects and containers comprised of data structures storing the objects subsequently transferred from the host to the object storage upon receipt of the second API command.

5. The system of claim 4 , wherein the at least one processor device excludes repository data from the candidate list based on at least one of the predetermined age threshold and repository data marked as excluded by an end user.

6. The system of claim 4 , wherein the at least one processor device maintains a mapping of the migrated certain deduplication repository data between the host and the object storage by updating host repository metadata.

7. A computer program product for managing data in a data deduplication repository in a computing storage environment, by a processor device, the computer program product embodied on a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:
an executable portion that migrates certain deduplication repository data from a host to an object storage by integrating a data offload process with an existing backup management application executing on the host, for reducing unnecessary deduplication repository data stored on the host; wherein the certain deduplication repository data comprises data identified by the existing backup management application as candidate repository data developed on a candidate list; and the candidate repository data on the candidate list includes repository data having a reference count number below a predetermined reference count threshold, the reference count number associated with a deduplication ratio; and
an executable portion that issues a first application programming interface (API) command, by the existing backup management application to a deduplication gateway having input specifying to locate any data within a subset of entities having the reference count number below the predetermined reference count threshold and a data age older than a predetermined age threshold; wherein upon receiving the first API command, the deduplication gateway scans metadata of the subset of entities in a deduplication repository to identify the candidate repository data according to the first API command input, the subset of entities comprising at least one of a range of virtual cartridges, all cartridges in a virtual library, and all cartridges in the deduplication repository; and
an executable portion that, responsive to receiving the candidate list by the backup management application, initiates the data offload process by migrating the cand

Assignees

• IBM

Inventors

• Dain Joseph W
• Kishi Gregory T
• Paz Gil E
• UGALDE AMEZCUA Renan Jeshua

Classifications

• G06F16/1748

  De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641) · CPC title

• G06F11/1451

  by selection of backup contents · CPC title

• G06F11/1453 Primary

  using de-duplication of the data · CPC title

• G06F16/1752

  based on file chunks · CPC title

• G06F3/0641

  De-duplication techniques · CPC title