Deduplicating data integrity checks across systems

Patent metadata

Field	Value
Publication number	US-11983147-B2
Application number	US-202117337086-A
Country	US
Kind code	B2
Filing date	Jun 2, 2021
Priority date	Jun 2, 2021
Publication date	May 14, 2024
Grant date	May 14, 2024

Abstract

A computer-implemented method, according to one embodiment, includes: receiving, at a clustered filesystem from a formatted filesystem, a request to perform a data integrity check for a portion of data. A determination is made as to whether the request includes a filesystem type of the portion of data, and in response to determining that the request includes a filesystem type of the portion of data, another determination is made as to whether the clustered filesystem supports the data integrity check for the filesystem type. In response to determining the clustered filesystem supports the data integrity check, another determination is made as to whether the portion of data is currently available. Furthermore, the computer-implemented method includes causing the data integrity check to be performed in response to determining that the portion of data is currently available. Results of performing the data integrity check are also sent to the formatted filesystem.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving, at a clustered filesystem from a formatted filesystem, a first request to perform a data integrity check for a portion of data;
determining whether the first request includes a filesystem type of the portion of data,
wherein determining whether the first request includes the filesystem type of the portion of data includes determining whether the portion of data has a metadata tag which corresponds to the portion of data;
in response to a determination that the portion of data does not have a metadata tag, sending instructions to a FSCK daemon to add a metadata tag on a file that includes the portion of data;
receiving a second request to perform the data integrity check for the portion of data, wherein the second request includes the metadata tag appended thereto, wherein the metadata tag appended to the second request is in a form of a file header;
extracting the metadata tag appended to the second request from the second request;
using the metadata tag extracted from the second request and the metadata tag added to the file for determining whether a second request includes the filesystem type of the portion of data,
in response to a determination that the second request includes the filesystem type of the portion of data, determining whether the clustered filesystem supports the data integrity check for the filesystem type of the portion of data;
in response to a determination the clustered filesystem supports the data integrity check for the filesystem type of the portion of data, determining whether the portion of data is currently available;
in response to a determination that the portion of data is currently available, causing the data integrity check to be performed by the clustered filesystem on the portion of data, wherein the data integrity check is not performed by the formatted filesystem; and
sending results of performing the data integrity check to the formatted filesystem.

2. The computer-implemented method of claim 1 , wherein the data integrity check is a FSCK operation, wherein the FSCK operation includes:
updating an allocation map to indicate blocks as free that have been incorrectly allocated, creating directory entries for files and/or directories that have inodes allocated but for which no directory entries exist,
removing directory entries that point to directory entries which include metadata that contradicts metadata stored in the respective inodes, and
updating link counts on files and/or directories to reflect accurate numbers.

3. The computer-implemented method of claim 2 , comprising:
establishing a communication channel between a FSCK process in the formatted filesystem and a FSCK process in the clustered filesystem,
wherein the first request is received from the formatted filesystem along the communication channel,
wherein causing the data integrity check to be performed by the clustered filesystem on the portion of data includes:
sending one or more instructions to the FSCK daemon to perform the data integrity check.

4. The computer-implemented method of claim 1 , wherein the results of performing the data integrity check that are sent to the formatted filesystem include FSCK timestamps.

5. The computer-implemented method of claim 1 , wherein determining whether the portion of data is currently available includes:
determining, from the metadata tag added to the file, whether the portion of data has been exported from the clustered filesystem and/or is currently used as local storage by the formatted filesystem.

6. The computer-implemented method of claim 1 , wherein the portion of data is determined to have a metadata tag which corresponds to the portion of data, wherein the determined metadata tag which corresponds to the portion of data includes information associated with the portion of data, wherein the information includes application container details.

7. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions are readable and/or executable by a processor to cause the processor to:
receive, by the processor at a backend clustered filesystem from a formatted filesystem, a request to perform a data integrity check for a portion of data;
determine, by the processor, whether an extended attribute appended to the request includes a filesystem type of the portion of data,
wherein the extended attribute is in a form of a file header;
in response to a determination that the extended attribute appended to the request includes a filesystem type of the portion of data, determine, by the processor, whether the backend clustered filesystem supports the data integrity check for the filesystem type of the portion of data;
in response to a determination that the backend clustered filesystem supports the data integrity check for the filesystem type of the portion of data, determine, by the processor, whether the portion of data is currently available;
in response to a determination that the portion of data is currently available, cause, by the processor, the data integrity check to be performed on the portion of data using the backend clustered filesystem, and in order to prevent operational duplicity, preventing the data integrity check from being performed in the formatted filesystem by not promoting the performance of the data integrity check in the formatted filesystem, wherein the data integrity check is a filesystem consistency check (FSCK) operation performed on a file that includes the portion of data; and
send, by the processor, results of performing the data integrity check to the formatted filesystem.

8. The computer program product of claim 7 , wherein the program instructions are readable and/or executable by the processor to cause the processor to:
establish, by the processor, a communication channel between a FSCK process in the formatted filesystem and a FSCK process in the backend clustered filesystem,
wherein the request is received from the formatted filesystem along the communication channel,
wherein causing the data integrity check to be performed on the portion of data includes:
sending one or more instructions to a FSCK daemon associated with the FSCK process in the backend clustered filesystem to perform the data integrity check.

9. The computer program product of claim 8 , wherein causing the data integrity check to be performed on the portion of data includes:
sending one or more instructions to an FSCK daemon to perform the FSCK operation.

10. The computer program product of claim 7 , wherein the results of performing the data integrity check include FSCK timestamps.

11. The computer program product of claim 7 , wherein determining whether the portion of data is currently available includes:
determining whether the portion of data has been exported from the backend clustered filesystem and/or is currently used as local storage by the formatted filesystem.

12. The computer program product of claim 7 , wherein the portion of data has a metadata tag which corresponds to the portion of data,
wherein the metadata tag includes information associated with the portion of data,
wherein the information includes filesystem type.

13. A system, comprising:
a processor; and
logic integrated with the processor, executable by the processor, or integrated with and executable by the processor, the logic being configured to:
receive, by the processor at a clustered filesystem of a first environment from a formatted filesystem of a second environment having data storage devices with data physically stored therein, a

Assignees

• IBM

Inventors

• Ali Asmahan
• Lara Christina
• Eda Sasikanth
• Jain Abhishek
• Gandhi Sanjay Vipin

Classifications

• G06F16/174 Primary

  Redundancy elimination performed by the file system (error detection or correction of the data by redundancy in operations G06F11/14) · CPC title

• G06F3/0608

  Saving storage space on storage systems · CPC title

• G06F3/0619

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F3/0641

  De-duplication techniques · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title