Systems and methods of platform-agnostic metadata analysis

Patent metadata

Field	Value
Publication number	US-11188525-B2
Application number	US-201916428256-A
Country	US
Kind code	B2
Filing date	May 31, 2019
Priority date	May 31, 2019
Publication date	Nov 30, 2021
Grant date	Nov 30, 2021

Abstract

A system and method of generating platform-dependent queries from a platform-agnostic query are disclosed. A data pipeline including a plurality of events having a set of platform-dependent metadata associated therewith is implemented. Each of the plurality of events is processed by one of a plurality of ingestion platforms. A platform-agnostic query configured to obtain one or more metadata search results from the platform-dependent metadata is received and a first platform-dependent query is generated from the platform-agnostic query. The first platform-dependent query is configured to be implemented by at least one target ingestion platform.

First claim

What is claimed is:

1. A system, comprising:
a computing device configured to:
implement a data pipeline comprising a plurality of events, wherein each event in the plurality of events has a set of platform-dependent metadata associated therewith, and wherein each of the plurality of events is processed by one of a plurality of ingestion platforms;
receive a platform-agnostic query configured to obtain one or more metadata search results from the platform-dependent metadata;
determine one or more events of the plurality of events based on the platform-agnostic query and the set of platform-dependent metadata associated with the one or more events;
identify at least one target ingestion platform from the plurality of ingestion platforms based at least in part on the determined one or more events, wherein at least one of the one or more events is processed by the at least one target ingestion platform; and
generate a first platform-dependent query from the platform-agnostic query based at least in part on one or more conversion rules associated with the at least one target ingestion platform, wherein the first platform-dependent query is configured to be implemented by at least one target ingestion platform.

2. The system of claim 1 , wherein the computing device is configured to generate the at least one platform-dependent query by applying the one or more conversion rules to the platform-agnostic query.

3. The system of claim 2 , wherein the computing device is configured to generate a second platform-dependent query from the platform-agnostic query by applying a second set of platform-dependent conversion rules to the platform-agnostic query, wherein the second set of platform-dependent conversion rules correspond to a second target ingestion platform.

4. The system of claim 1 , wherein the computing device is configured to deploy the first platform-dependent query to the at least one target ingestion platform.

5. The system of claim 1 , wherein the computing device is configured to parse the platform-agnostic query.

6. The system of claim 1 , wherein the computing device is configured to identify the at least one target ingestion platform by executing the platform-agnostic query in a combined metadata repository.

7. The system of claim 1 , wherein the platform-agnostic query comprises an SQL query.

8. The system of claim 1 , wherein the platform-dependent query is a platform-dependent metric calculator.

9. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by a processor cause a device to perform operations comprising:
implementing a data pipeline comprising a plurality of events, wherein each event in the plurality of events has a set of platform-dependent metadata associated therewith, and wherein each of the plurality of events is processed by one of a plurality of ingestion platforms;
receiving a platform-agnostic query configured to obtain one or more metadata search results from the platform-dependent metadata;
determining one or more events of the plurality of events based on the platform-agnostic query and the set of platform-dependent metadata associated with the one or more events;
identifying at least one target ingestion platform from the plurality of ingestion platforms based at least in part on the determined one or more events, wherein at least one of the one or more events is processed by the at least one target ingestion platform; and
generating a first platform-dependent query from the platform-agnostic query based at least in part on one or more conversion rules associated with the at least one target ingestion platform, wherein the first platform-dependent query is configured to be implemented by at least one target ingestion platform.

10. The non-transitory computer readable medium of claim 9 , wherein generating the at least one platform-dependent query comprises applying the one or more conversion rules to the platform-agnostic query.

11. The non-transitory computer readable medium of claim 10 , wherein the instructions when executed by the processor cause the device to perform further operations comprising:
generating a second platform-dependent query from the platform-agnostic query by applying a second set of platform-dependent conversion rules to the platform-agnostic query, wherein the second set of platform-dependent conversion rules correspond to a second target ingestion platform.

12. The non-transitory computer readable medium of claim 9 , wherein the instructions when executed by the processor cause the device to perform further operations comprising deploying the first platform-dependent query to the at least one target ingestion platform.

13. The non-transitory computer readable medium of claim 9 , wherein the instructions when executed by the processor cause the device to perform further operations comprising parsing the platform-agnostic query.

14. The non-transitory computer readable medium of claim 9 , wherein the instructions when executed by the processor cause the device to perform further operations comprising identifying the at least one target ingestion platform by executing the platform-agnostic query in a combined metadata repository.

15. The non-transitory computer readable medium of claim 9 , wherein the platform-agnostic query comprises an SQL query.

16. The non-transitory computer readable medium of claim 9 , wherein the platform-dependent query is a platform-dependent metric calculator.

17. A method, comprising:
implementing a data pipeline comprising a plurality of events, wherein each event in the plurality of events has a set of platform-dependent metadata associated therewith, and wherein each of the plurality of events is processed by one of a plurality of ingestion platforms;
receiving a platform-agnostic query configured to obtain one or more metadata search results from the platform-dependent metadata;
determining one or more events of the plurality of events based on the platform-agnostic query and the set of platform-dependent metadata associated with the one or more events;
identifying at least one target ingestion platform from the plurality of ingestion platforms based at least in part on the determined one or more events, wherein at least one of the one or more events is processed by the at least one target ingestion platform; and
generating a first platform-dependent query from the platform-agnostic query based at least in part on one or more conversion rules associated with the at least one target ingestion platform, wherein the first platform-dependent query is configured to be implemented by at least one target ingestion platform.

18. The method of claim 17 , wherein generating the at least one platform-dependent query comprises applying the one or more conversion rules to the platform-agnostic query.

19. The method of claim 18 , comprising generating a second platform-dependent query from the platform-agnostic query by applying a second set of platform-dependent conversion rules to the platform-agnostic query, wherein the second set of platform-dependent conversion rules correspond to a second target ingestion platform.

20. The method of claim 17 , comprising deploying the first platform-dependent query to the at least one target ingestion platform.

Assignees

• Walmart Apollo Llc

Inventors

• Athavale Vilas
• Deshmukh Swati Ajit
• Matte Ravinder
• Viswanathan Arun
• Gupta Preksha

Classifications

• G06F16/2458

  Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries · CPC title

• G06F16/908 Primary

  using metadata automatically derived from the content · CPC title

• G06F16/2425 Primary

  Iterative querying; Query formulation based on the results of a preceding query · CPC title