Extensible, secure and efficient monitoring and diagnostic pipeline for hybrid cloud architecture

Patent metadata

Field	Value
Publication number	US-11765058-B2
Application number	US-202217572946-A
Country	US
Kind code	B2
Filing date	Jan 11, 2022
Priority date	May 18, 2018
Publication date	Sep 19, 2023
Grant date	Sep 19, 2023

Abstract

Systems are provided for facilitating the disclosed methods for performing event storage and diagnostic processing within a hybrid cloud environment. Event records are gathered and batched at an on-premises server. The event records are also appended with correlation vector data that enables the event records to be correlated with other events. The batch of event record batches are signed with a security key associated with a cloud storage container and the on-premises server is restricted to writing the batch of event records to the container. In some instances, the size of the batch is based on a duration of time for collecting records, which can be adjusted to accommodate for missing data.

First claim

What is claimed is:

1. A system comprising:
a processor; and
memory coupled to the processor, the memory comprising computer executable instructions that, when executed by the processor, performs a method comprising:
generating event records corresponding to monitoring events identified in an on-premises computing system, wherein generating the event records comprises:
determining a first configuration format for a first event record in the event records based on a type of monitoring event associated with the first record; and
determining a second configuration format for a second event record in the event records based on a type of monitoring event associated with the second record;

signing the event records with a security key identifying a storage container operable to store the signed event records, the storage container being managed by a cloud storage in communication with the on-premises computing system; and
transmitting the signed event records to the cloud storage, a security key signature of the event records providing access to the storage container for storing the signed event records.

2. The system of claim 1 , wherein first configuration format corresponds to a telemetry event, the telemetry event representing a low frequency event.

3. The system of claim 1 , wherein second configuration format corresponds to a diagnostic event, the diagnostic event representing a high frequency event.

4. The system of claim 1 , wherein the cloud storage manages a plurality of storage containers, each storage container of the plurality of storage containers being isolated from other storage containers of the plurality of storage containers.

5. The system of claim 1 , wherein transmitting the signed event records to the cloud storage comprises transmitting an explicit identifier of the storage container to the cloud storage.

6. The system of claim 1 , wherein transmitting the signed event records to the cloud storage comprises transmitting the security key to the cloud storage.

7. The system of claim 1 , wherein the security key signature of the event records is used to identify the storage container based on a determined correlation between the security key and a corresponding security key, the corresponding security key being stored by the cloud storage and being associated with the storage container.

8. The system of claim 1 , wherein the method further comprises:
receiving, by the cloud system, the signed event records; and
evaluating, by the cloud system, content of the signed event records to determine whether the content matches at least one of:
an expected format type;
an expected size requirement; or
an expected flow pattern.

9. The system of claim 1 , wherein generating the event records further comprises:
applying a first correlation vector data to the first event record, the first correlation vector data identifying a first context associated with the first event record; and
applying a second correlation vector data to the second event record, the second correlation vector data identifying a second context associated with the second event record.

10. The system of claim 1 , wherein the method further comprises:
after generating the event records, batching the event records into one or more batches based on at least one of:
a type of monitoring event; or
a type of device reporting a monitoring event.

11. The system of claim 10 , wherein batching the event records comprises serializing the first event record into a compressed format and serializing the second event record into an uncompressed format.

12. The system of claim 1 , wherein the security key restricts access to the storage container to write access.

13. The system of claim 12 , wherein the security key provides no access to other storage containers managed by the cloud storage.

14. The system of claim 12 , wherein the security key provides access to one or more other storage containers managed by the cloud storage.

15. A method comprising:
generating event records corresponding to monitoring events identified in an on-premises computing system, wherein generating the event records comprises:
determining a first configuration format for a first event record in the event records based on a type of monitoring event associated with the first record; and
determining a second configuration format for a second event record in the event records based on a type of monitoring event associated with the second record;

signing the event records with a security key identifying a storage container operable to store the signed event records, the storage container being managed by a cloud storage in communication with the on-premises computing system; and
transmitting the signed event records to the cloud storage, a security key signature of the event records providing access to the storage container for storing the signed event records.

16. The method of claim 15 , wherein at least one of the first configuration format for the first event record or the second configuration format for the second event record is further based on a frequency of a corresponding monitoring event.

17. The method of claim 15 , wherein the security key is associated with a corresponding security key being stored by the cloud storage and restricting access to the storage container.

18. The method of claim 15 , wherein content of the signed event records is evaluated to determine whether the content matches at least one of:
records known to be created by a device being monitored; or
records matching an attribute previously communicated to the cloud storage.

19. The method of claim 15 , wherein the security key signature is created by signing the event records with the security key.

20. A device comprising:
a processor; and
memory coupled to the processor, the memory comprising computer executable instructions that, when executed by the processor, performs a method comprising:
generating event records corresponding to monitoring events identified in an on-premises computing system;
signing the event records with a security key identifying a storage container operable to store the signed event records, the event records comprising a first event record and a second event record, the first event record having a first configuration format based on a first type of monitoring event associated with the first event record, the second event record having a second configuration format based on a second type of monitoring event associated with the second event record, the storage container being managed by a cloud storage in communication with the on-premises computing system; and
transmitting the signed event records to the cloud storage, a security key signature of the event records providing access to the storage container for storing the signed event records.

Assignees

• Microsoft Technology Licensing Llc
• Microsoft Tech Licesning Llc

Inventors

• Bharatkumar Bhatt Rutwick
• Mohit Asudani

Classifications

• G06F11/3006 Primary

  where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50) · CPC title

• H04L43/067 Primary

  using time frame reporting · CPC title

• G06F11/3075

  the data filtering being achieved in order to maintain consistency among the monitored data, e.g. ensuring that the monitored data belong to the same timeframe, to the same system or component · CPC title

• G06F16/178

  Techniques for file synchronisation in file systems · CPC title

• G06F21/6218

  to a system of files or objects, e.g. local or distributed file system or database · CPC title