Systems and methods for backup and restore of container-based persistent volumes

Patent metadata

Field	Value
Publication number	US-12032453-B2
Application number	US-201916599752-A
Country	US
Kind code	B2
Filing date	Oct 11, 2019
Priority date	Oct 11, 2019
Publication date	Jul 9, 2024
Grant date	Jul 9, 2024

Abstract

Methods and systems for backing up and restoring data on a worker node are described. According to some embodiments, the method includes using a job controller to spin up one or more backup jobs in the worker node. The method further includes utilizing, by the backup jobs, mount propagation to access one or more persistent volumes of a pod on the worker node. The method further includes performing, by the backup jobs, backup tasks on the persistent volumes. The method further includes applying a set of rules used by a scheduler to determine a location of the pod in order to ensure the backup jobs are scheduled on a same worker node. The method further includes storing, by the backup jobs, backup artifacts generated by the backup tasks, wherein the backup artifacts include deduplicated data.

First claim

What is claimed is:

1. A computer-implemented method of backing up data on a worker node among a plurality of worker nodes, comprising:
using a job controller to spin up one or more backup jobs in the worker node;
utilizing, by the backup jobs, mount propagation to access one or more persistent volumes of a pod, among a plurality of pods, on the worker node;
performing, by the backup jobs, backup tasks on the persistent volumes of the pod;
applying a set of rules used by a scheduler to determine a location of the pod to ensure the backup jobs are scheduled on a same worker node where the pod is running, wherein the set of rules is defined based on custom labels on the plurality of worker nodes and label sectors specified in the plurality of pods; and
storing, by the backup jobs, backup artifacts generated by the backup tasks, wherein the backup artifacts include deduplicated data, wherein, prior to performing the backup tasks on the persistent volumes of the pod, the method further includes:
using pre-hooks to freeze a file system of the worker node, wherein the persistent volumes are mapped on the file system of the worker node; and
cleaning up the backup jobs based on reaching a specified number of successful completions.

2. The method of claim 1 , further comprising:
after performing the backup tasks on the persistent volumes of the pod, unfreezing the file system of the worker node.

3. The method of claim 2 , further comprising:
collecting metadata associated with the backup tasks performed on the persistent volumes.

4. The method of claim 1 , further comprising: prior to storing the backup artifacts generated by the backup tasks, sending the backup artifacts to a data server to store the backup artifacts.

5. The method of claim 1 , wherein a value of the mount propagation is HostToContainer or Bidirectional.

6. The method of claim 1 , wherein the job controller is a kube-controller-manager.

7. The method of claim 1 , wherein the set of rules used by the scheduler is node affinity.

8. The method of claim 1 , wherein the backup artifacts further include logging information, configuration information across containers consisted in the pod, and storing secrets.

9. A computer-implemented method of restoring data on a worker node among a plurality of worker nodes, comprising:
using a job controller to spin up one or more restore jobs in the worker node;
utilizing, by the restore jobs, mount propagation to access one or more persistent volumes of a pod, among a plurality of pods, on the worker node;
applying a set of rules used by a scheduler to determine a location of the pod to ensure the restore jobs are scheduled on a same worker node where the pod is running, wherein the set of rules is defined based on custom labels on the plurality of worker nodes and label sectors specified in the plurality of pods;
retrieving, by the restore jobs, backup artifacts; and
performing, by the restore jobs, restore tasks on the persistent volumes of the pod based on the retrieved backup artifacts, wherein prior to performing the restore tasks on the persistent volumes of the pod, the method includes:
using pre-hooks to freeze a file system of the worker node, wherein the persistent volumes are mapped on the file system of the worker node; and
cleaning up the restore jobs based on reaching a specified number of successful completions.

10. The method of claim 9 , further comprising:
after performing the restore tasks on the persistent volumes of the pod, unfreezing the file system of the worker node.

11. The method of claim 10 , further comprising:
collecting metadata associated with the restore tasks performed on the persistent volumes.

12. The method of claim 9 , wherein the backup artifacts are retrieved from a data server.

13. The method of claim 9 , wherein a value of the mount propagation is HostToContainer or Bidirectional.

14. The method of claim 9 , wherein the job controller is a kube-controller-manager.

15. The method of claim 9 , wherein the set of rules used by the scheduler is node affinity.

16. The method of claim 9 , wherein the backup artifacts include deduplicated data, logging information, configuration information across containers consisted in the pod, and storing secrets associated with the persistent volumes.

17. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:
using a job controller to spin up one or more backup jobs in a worker node among a plurality of worker nodes;
utilizing, by the backup jobs, mount propagation to access one or more persistent volumes of a pod, among a plurality of pods, on the worker node;
performing, by the backup jobs, backup tasks on the persistent volumes of the pod;
applying a set of rules used by a scheduler to determine a location of the pod to ensure the backup jobs are scheduled on a same worker node where the pod is running, wherein the set of rules is defined based on custom labels on the plurality of worker nodes and label sectors specified in the plurality of pods; and
storing, by the backup jobs, backup artifacts generated by the backup tasks, wherein the backup artifacts include deduplicated data, wherein, prior to performing the backup tasks on the persistent volumes of the pod, the operations further comprising:
using pre-hooks to freeze a file system of the worker node, wherein the persistent volumes are mapped on the file system of the worker node; and
cleaning up the backup jobs based on reaching a specified number of successful completions.

18. A data processing system, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including:
using a job controller to spin up one or more backup jobs in a worker node among a plurality of worker nodes;
utilizing, by the backup jobs, mount propagation to access one or more persistent volumes of a pod, among a plurality of pods, on the worker node;
performing, by the backup jobs, backup tasks on the persistent volumes of the pod;
applying a set of rules used by a scheduler to determine a location of the pod to ensure the backup jobs are scheduled on a same worker node where the pod is running, wherein the set of rules is defined based on custom labels on the plurality of worker nodes and label sectors specified in the plurality of pods; and
storing, by the backup jobs, backup artifacts generated by the backup tasks, wherein the backup artifacts include deduplicated data, wherein, prior to performing the backup tasks on the persistent volumes of the pod, the operations further comprising:
using pre-hooks to freeze a file system of the worker node, wherein the persistent volumes are mapped on the file system of the worker node; and
cleaning up the backup jobs based on reaching a specified number of successful completions.

19. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:
using a job controller to spin up one or more restore jobs in a worker node among a plurality of worker nodes;
utilizing, by the restore jobs, mount propagation to access one or more persistent volumes of a pod, among a plurality of pods, on the worker node;
applying a set of rules used by a scheduler to determine a location of the pod to ensure

Assignees

• Emc Ip Holding Co Llc

Inventors

• Al-Alem Luay
• Bett Antony

Classifications

• G06F11/1464

  for networked environments · CPC title

• G06F16/215

  Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors · CPC title

• G06F2201/84

  Using snapshots, i.e. a logical point-in-time copy of the data · CPC title

• G06F11/1453

  using de-duplication of the data · CPC title

• G06F11/1469 Primary

  Backup restoration techniques · CPC title