Multi-datacenter message queue

Patent metadata

Field	Value
Publication number	US-2016337465-A1
Application number	US-201615154141-A
Country	US
Kind code	A1
Filing date	May 13, 2016
Priority date	May 15, 2015
Publication date	Nov 17, 2016
Grant date	—

Abstract

Approaches are disclosed for distributing messages across multiple data centers where the data centers do not store messages using a same message queue protocol. In some embodiment, a network element translates messages from a message queue protocol (e.g., Kestrel, RABBITMQ, APACHE Kafka, and ACTIVEMQ) to an application layer messaging protocol (e.g., XMPP, MQTT, WebSocket protocol, or other application layer messaging protocols). In other embodiments, a network element translates messages from an application layer messaging protocol to a message queue protocol. Using the new approaches disclosed herein, data centers communicate using, at least in part, application layer messaging protocols to disconnect the message queue protocols used by the data centers and enable sharing messages between messages queues in the data centers. Consequently, the data centers can share messages regardless of whether the underlying message queue protocols used by the data centers (and the network devices therein) are compatible with one another.

First claim

What is claimed is:

1 . A method comprising:
receiving from a first message queue, a message encoded in a first message queue protocol associated with the first message queue; generating a translated message by translating the message from the first message queue protocol to an application layer messaging protocol; and transmitting, utilizing the application layer messaging protocol, the translated message to a server, wherein the translated message update a second message queue, the second message queue being encoded in a second message queue protocol.

2 . The method of claim 1 , further comprising:
determining whether a connection to the server is active; and dynamically adjusting a size of a buffer for the first message queue based on whether the connection to the server is active.

3 . The method of claim 2 , wherein the dynamically adjusting the size of the buffer for the first message queue based on whether the connection to the server is active comprises:
when it is determined that the connection is active, increasing the size of the buffer, and when it is determined that the connection is not active, decreasing the size of the buffer.

4 . The method of claim 2 , wherein the determining whether a connection to the server is active comprises receiving, from the server, an instruction to stop transmitting messages associated with the second message queue.

5 . The method of claim 4 , further comprising:
when it is determined that the connection is not active:
storing messages in the buffer, and
stopping transmission to the server of messages associated with the second message queue;
when it is determined that the connection is active: transmitting, to the server from the buffer, messages associated with the second message queue.

6 . The method of claim 2 , wherein each of the first message queue protocol and the second message queue protocol is a message queue protocol selected from the group consisting of Kestrel, RABBITMQ, APACHE Kafka, and ACTIVEMQ.

7 . The method of claim 6 , wherein the first queue protocol is a different message queue protocol than the second queue protocol.

8 . The method of claim 1 , wherein the translated message by translating the message from the first message queue protocol to the application layer messaging protocol comprises:
translating the message from an APACHE Kafka message encoding to an Extensible Messaging and Presence Protocol (XMPP) message encoding.

9 . An apparatus comprising:
a network interface, the network interface being configured to receive, from a first message queue, a message encoded in a first message queue protocol associated with the first message queue; a memory element to store code; at least one processor coupled to the memory element and network interface, wherein the at least one processor is to execute the code to perform operations comprising:
generating a translated message by translating the message from the first message queue protocol to an application layer messaging protocol; and
transmitting, utilizing the application layer messaging protocol and the network interface, the translated message to a server, wherein the translated message is to update a second message queue encoded in a second message queue protocol.

10 . The apparatus of claim 9 , further comprising:
a buffer configured to store messages, and wherein the operations further comprise:
determining whether a connection to the server is active; and
dynamically adjusting a size of a buffer for the first message queue based on whether the connection to the server is active.

11 . The apparatus of claim 10 , wherein the dynamically adjusting the size of the buffer for the first message queue based on whether the connection to the server is active comprises:
when it is determined that the connection is active, increasing the size of the buffer, and when it is determined that the connection is not active, decreasing the size of the buffer.

12 . The apparatus of claim 9 , wherein the determining whether the connection to the server is active comprises receiving, from the server, an instruction to stop transmitting messages associated with the second message queue.

13 . The apparatus of claim 12 , wherein the operations further comprise:
when it is determined that the connection is not active:
stopping transmission to the server of messages associated with the second message queue, and
storing, in the buffer, the messages associated with the second message queue; and
when it is determined that the connection is active: transmitting, to the server from the buffer, the messages associated with the second message queue.

14 . The apparatus of claim 9 , wherein the generating the translated message by translating the message from the first message queue protocol to the application layer messaging protocol comprises:
translating the message from a APACHE Kafka message encoding to an Extensible Messaging and Presence Protocol (XMPP) message encoding.

15 . A computer-readable non-transitory medium comprising instructions, that when executed by at least one processor configure the at least one processor to perform operations comprising:
receiving from a first message queue, a message encoded in a first message queue protocol associated with the first message queue; generating a translated message by translating the message from the first message queue protocol to an application layer messaging protocol; and transmitting, utilizing the application layer messaging protocol, the translated message to a server, wherein the translated message update a second message queue, the second message queue being encoded in a second message queue protocol.

16 . The computer-readable non-transitory medium of claim 15 , further comprising:
determining whether a connection to the server is active; and dynamically adjusting a size of a buffer for the first message queue based on whether the connection to the server is active.

17 . The computer-readable non-transitory medium of claim 16 , wherein the dynamically adjusting the size of the buffer for the first message queue based on whether the connection to the server is active comprises:
when it is determined that the connection is active, increasing the size of the buffer, and when it is determined that the connection is not active, decreasing the size of the buffer.

18 . The computer-readable non-transitory medium of claim 16 , wherein the determining whether a connection to the server is active comprises receiving, from the server, an instruction to stop transmitting messages associated with the second message queue.

19 . The computer-readable non-transitory medium of claim 18 , further comprising:
when it is determined that the connection is not active:
storing messages in the buffer, and
stopping transmission to the server of messages associated with the second message queue;
when it is determined that the connection is active: transmitting, to the server, from the buffer, messages associated with the second message queue.

20 . The computer-readable non-transitory medium of claim 15 , wherein the translated message by translating the message from the first message queue protocol to the application layer messaging protocol comprises:
translating the message from a APACHE Kafka message encoding to an Extensible Messaging and Presence Protocol (XMPP) message encoding.

Assignees

• Cisco Tech Inc

Inventors

• Tarre Marc Solanas
• Rantzau Ralf
• Dutta Debojyoti
• Sharma Manoj

Classifications

• G06F9/00

  Arrangements for program control, e.g. control units (program control for peripheral devices G06F13/10) · CPC title

• H04L49/9005

  using dynamic buffer space allocation · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• H04L67/2823 Primary

  Electricity · mapped topic

• H04L69/329

  in the application layer [OSI layer 7] · CPC title