Techniques and architectures for recovering from a service disruption in a multi-server environment

Patent metadata

Field	Value
Publication number	US-10958505-B2
Application number	US-201916532352-A
Country	US
Kind code	B2
Filing date	Aug 5, 2019
Priority date	May 11, 2017
Publication date	Mar 23, 2021
Grant date	Mar 23, 2021

Abstract

Techniques and architectures to provide multiple application servers each having at least one connection agent and at least one enabler agent and at least one database organized as multiple nodes. An error condition between a first connection agent and a first database node is detected. The first connection agent indicates that the first database node is unavailable to stop attempts to access the first database node by a first application server. A first enabler agent periodically checks availability of the first database node. The first database node is indicated as available in response to a pre-selected number of successful attempts to contact the first database node with the first enabler agent. The first database node is accessed with the application server in response to the indication that the first database node is available.

First claim

What is claimed is:

1. A method in a computing environment having multiple application servers each having at least one connection agent and at least one enabler agent, the computing environment to be executed by one or more hardware processing components coupled with at least one memory device, the computing environment further having at least one database organized as multiple nodes, the method comprising:
detecting an error condition between a first connection agent and a first database node;
indicating, with the first connection agent, that the first database node is unavailable to stop attempts to access the first database node by a first application server;
checking periodically, with a first enabler agent, availability of the first database node;
causing the first database node to be restarted in response to a pre-selected number of unsuccessful attempts to contact the first database node with the first enabler agent;
accessing the first database node with the application server in response to the indication that the first database node is available.

2. The method of claim 1 wherein the error condition comprises a connection between the first connection agent and the first database node being non-responsive.

3. The method of claim 1 wherein the error condition comprises the first database node being unavailable.

4. The method of claim 1 , wherein the computing environment further comprises at least a second enabler agent, a second connection agent and a second application server that operate independently of the first enabler agent, the first connection agent and the first application server, the method comprising:
detecting an error condition between the second connection agent and the first database node;
indicating, with the second connection agent, that the first database node is unavailable to stop attempts to access the first database node by the second application server;
checking periodically, with the second enabler agent, availability of the first database node;
causing the first database node to be restarted in response to a pre-selected number of unsuccessful attempts to contact the first database node with the second enabler agent;
accessing the first database node with the second application server in response to the indication that the first database node is available.

5. A non-transitory computer-readable medium to provide, in a computing environment having multiple application servers each having at least one connection agent and at least one enabler agent, the computing environment to be executed by one or more hardware processing components coupled with at least one memory device, the computing environment further having at least one database organized as multiple nodes, instructions that, when executed by one or more processors, are configurable to cause the one or more processors to:
detect an error condition between a first connection agent and a first database node;
indicate, with the first connection agent, that the first database node is unavailable to stop attempts to access the first database node by a first application server;
check periodically, with a first enabler agent, availability of the first database node;
cause the first database node to be restarted in response to a pre-selected number of unsuccessful attempts to contact the first database node with the first enabler agent;
access the first database node with the application server in response to the indication that the first database node is available.

6. The non-transitory computer-readable medium of claim 5 wherein the error condition comprises a connection between the first connection agent and the first database node being non-responsive.

7. The non-transitory computer-readable medium of claim 5 wherein the error condition comprises the first database node being unavailable.

8. The non-transitory computer-readable medium of claim 5 , wherein the computing environment further comprises at least a second enabler agent, a second connection agent and a second application server that operate independently of the first enabler agent, the first connection agent and the first application server, the instructions, when executed by the one or more processors, are configurable to cause the one or more processors to:
detect an error condition between the second connection agent and the first database node;
indicate, with the second connection agent, that the first database node is unavailable to stop attempts to access the first database node by the second application server;
check periodically, with the second enabler agent, availability of the first database node;
cause the first database node to be restarted in response to a pre-selected number of unsuccessful attempts to contact the first database node with the second enabler agent;
access the first database node with the second application server in response to the indication that the first database node is available.

9. A system comprising a computing environment having multiple application servers each having at least one connection agent and at least one enabler agent, the computing environment to be executed by one or more hardware processing components coupled with at least one memory device, the computing environment further having at least one database organized as multiple nodes, the system having one or more processors configurable to:
detect an error condition between a first connection agent and a first database node;
indicate, with the first connection agent, that the first database node is unavailable to stop attempts to access the first database node by a first application server;
check periodically, with a first enabler agent, availability of the first database node;
cause the first database node to be restarted in response to a pre-selected number of unsuccessful attempts to contact the first database node with the first enabler agent;
access the first database node with the application server in response to the indication that the first database node is available.

10. The system of claim 9 wherein the error condition comprises a connection between the first connection agent and the first database node being non-responsive.

11. The system of claim 9 wherein the error condition comprises the first database node being unavailable.

12. The system of claim 9 , wherein the computing environment further comprises at least a second enabler agent, a second connection agent and a second application server that operate independently of the first enabler agent, the first connection agent and the first application server, the system further configurable to:
detect an error condition between the second connection agent and the first database node;
indicate, with the second connection agent, that the first database node is unavailable to stop attempts to access the first database node by the second application server;
check periodically, with the second enabler agent, availability of the first database node;
cause the first database node to be restarted in response to a pre-selected number of unsuccessful attempts to contact the first database node with the second enabler agent;
access the first database node with the second application server in response to the indication that the first database node is available.

Assignees

• Salesforce Com Inc

Inventors

• Obembe Olumayokun
• Liu Chen
• Hsieh Ping-Hsiu
• Bijwe Ashwini
• Devadhar Vijayanth
• Chainani Mikhail
• Krishnan Sridevi Gopala
• Arbizu Alan
• Collins Jesse

Classifications

• G06F11/0793

  Remedial or corrective actions (recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40) · CPC title

• G06F11/0757

  by exceeding a time limit, i.e. time-out, e.g. watchdogs · CPC title

• G06F11/0709

  in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems · CPC title

• H04L41/0654 Primary

  using network fault recovery (ring fault isolation or reconfiguration in loop networks without recovery actions by a network management system H04L12/437) · CPC title