Self-healing architecture for resilient computing services

Patent metadata

Field	Value
Publication number	US-11522904-B2
Application number	US-201916712576-A
Country	US
Kind code	B2
Filing date	Dec 12, 2019
Priority date	May 25, 2017
Publication date	Dec 6, 2022
Grant date	Dec 6, 2022

Abstract

For each respective virtual machine (VM) of a plurality of VMs, a distributed computing system generates a unique Application Binary Interface (ABI) for an operating system for the respective VM, compiles a software application to use the unique ABI, and installs the operating system and the compiled software application on the respective VM. A dispatcher node dispatches, to one or more VMs of the plurality of VMs that provide a service and are in the active mode, request messages for the service. Furthermore, a first host device may determine, in response to software in the first VM invoking a system call in a manner inconsistent with the unique ABI for the operating system of the first VM, that a failover event has occurred. Responsive to the failover event, the distributed computing system fails over from the first VM to a second VM.

First claim

What is claimed is:

1. A method comprising:
initializing, by a distributed computing system, a plurality of virtual machines (VMs) configured to provide a service, wherein:
the distributed computing system hosts a first VM of the plurality of VMs,
the distributed computing system hosts a second VM of the plurality of VMs,
the first VM is in an active mode and the second VM is not in the active mode,
initializing the plurality of VMs comprises, for each respective VM of the plurality of VMs:
generating a randomized instance of a software component for the respective VM using address-space layout randomization such that the randomized instance of the software component for the respective VM includes a set of one or more non-executable gaps; and
installing the randomized instance of the software component for the respective VM on the respective VM;

dispatching, by the distributed computing system, to one or more VMs of the plurality of VMs that provide the service and are in the active mode, one or more request messages for the service;
determining, by the distributed computing system, in response to software in the first VM attempting to access data in the set of one or more non-executable gaps, that a failover event has occurred; and
responsive to determining that the failover event has occurred, failing over, by the distributed computing system, from the first VM to the second VM such that the second VM is in the active mode instead of the first VM.

2. The method of claim 1 , wherein the software component is an operating system kernel.

3. The method of claim 1 , wherein the software component is a software application.

4. The method of claim 1 , wherein the software component is a library instance.

5. The method of claim 1 , wherein:
the set of one or more non-executable gaps is a first set of one or more non-executable gaps,
the software component is a first software component,
the first software component and a second software component are different ones of: a operating system kernel, a software application, or a library,
initializing the respective VM further comprises generating a randomized instance of the second software component for the respective VM using address-space layout randomization such that the randomized instance of the second software component includes a second set of one or more non-executable gaps, and
determining that the failover event has occurred comprises determining, by the distributed computing system, in response to the software in the first VM attempting to access the data in the first set of one or more non-executable gaps or data in the second set of one or more non-executable gaps, that the failover event has occurred.

6. The method of claim 1 , wherein:
the distributed computing system comprises a first cluster and a second cluster, the first cluster including a first host device and a first persistent storage node, the second cluster including a second host device and a second persistent storage node,
the first host device hosts the first VM,
the second host device hosts the second VM;
the method further comprising:
prior to determining the failover event has occurred:
storing, at the first persistent storage node, a first copy of a database, the first persistent storage node persisting the first copy of the database independently of the first VM;
storing, at the second persistent storage node, a second copy of the database, the second persistent storage node persisting the second copy of the database independently of the second VM;
receiving, by the first VM, a request message for the service;
as part of processing the request message, generating, by the first VM, a database modification request;
responsive to the database modification request, performing, at the first persistent storage node, a modification to the first copy of the database; and
synchronizing the first copy and the second copy of the database; and

responsive to determining the failover event has occurred, rolling back the modification to the database requested by the first VM.

7. The method of claim 1 , wherein the method further comprises:
automatically discarding and replacing, by the distributed computing system, VMs of the plurality of VMs according to a refresh policy.

8. The method of claim 1 , further comprising:
responsive to determining that the failover event has occurred, discarding, by the distributed computing system, the first VM.

9. A distributed computing system comprising:
a dispatcher node; and
one or more host devices configured to:
initialize a plurality of virtual machines (VMs) configured to provide a service, wherein:
the plurality of VMs includes a first VM and a second VM,
the first VM is in an active mode and the second VM is not in the active mode,
wherein the one or more host devices are configured such that, as part of initializing the plurality of VMs, the one or more host devices, for each respective VM of the plurality of VMs:
generate a randomized instance of a software component for the respective VM using address-space layout randomization such that the randomized instance of the software component for the respective VM includes a set of one or more non-executable gaps; and
install the randomized instance of the software component for the respective VM on the respective VM;

wherein the dispatcher node is configured to distribute to one or more VMs of the plurality of VMs that provide the service and are in the active mode, one or more request messages for the service;
wherein the distributed computing system is configured to determine, in response to software in the first VM attempting to access data in the set of one or more non-executable gaps, that a failover event has occurred; and
wherein, responsive to determining that the failover event has occurred, the distributed computing system fails over from the first VM to the second VM such that the second VM is in the active mode instead of the first VM.

10. The distributed computing system of claim 9 , wherein the software component is an operating system kernel.

11. The distributed computing system of claim 9 , wherein the software component is a software application.

12. The distributed computing system of claim 9 , wherein the software component is a library instance.

13. The distributed computing system of claim 9 , wherein:
the set of one or more non-executable gaps is a first set of one or more non-executable gaps,
the software component is a first software component,
the first software component and a second software component are different ones of: a operating system kernel, a software application, or a library,
the one or more host devices are configured such that, as part of initializing the respective VM, the one or more host devices generate a randomized instance of the second software component for the respective VM using address-space layout randomization such that the randomized instance of the second software component includes a second set of one or more non-executable gaps, and
the distributed computing system is configured such that, as part of determining that the failover event has occurred, the distributed computing system determines, in response to the software in the first VM attempting to access the data in the first set of one or more non-executable gaps or data in the second set of one or more non-executable gaps, that the failover event has occurred.

14. The distributed computing system of claim 9 , wherein:
the distributed computing system comprises a first cluster and a second cluster, the first cluster including a first host device and a firs

Assignees

• Architecture Tech Corp

Inventors

• Mcardle Daniel
• Powers Judson
• Joyce Robert A

Classifications

• G06F11/1484

  involving virtual machines · CPC title

• G06F11/1446

  Point-in-time backing up or restoration of persistent data · CPC title

• H04L63/1441 Primary

  Countermeasures against malicious traffic (countermeasures against attacks on cryptographic mechanisms H04L9/002) · CPC title

• G06F2009/4557

  Distribution of virtual machine instances; Migration and load balancing · CPC title

• G06F11/202

  where processing functionality is redundant (redundant communication control functionality G06F11/2005, redundant storage control functionality G06F11/2089) · CPC title