Systems and methods for integration of multiple programming languages within a pipelined search query

Patent metadata

Field	Value
Publication number	US-11567735-B1
Application number	US-202017074280-A
Country	US
Kind code	B1
Filing date	Oct 19, 2020
Priority date	Oct 19, 2020
Publication date	Jan 31, 2023
Grant date	Jan 31, 2023

Abstract

According to one embodiment, a method that supports queries deploying operators based on multiple programming languages is described. A sequence of operators associated with a query is identified, where the sequence of operators includes at least two neighboring operators including a first operator based on a first programming language and a second operator based on a second programming language that is different from the first programming language. Thereafter, a schema associated with the first operator and a schema associated with the second operator is determined along with the compatibility between the schema of the first operator and the schema of the second operator. A query error message is generated in response to incompatibility between the first operator schema and the second operator schema. Compatibility is determined when an output generated by execution of the first operator provides machine data needed as input for execution of the second operator.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
obtaining a query from a web-based application in response to execution of a code cell corresponding to a plurality of statements in multiple programming languages including a first programming language and a second programming language;
identifying a sequence of operators associated with the query, the sequence of operators includes at least two neighboring operators including a first operator based on the first programming language and a second operator based on the second programming language that is different from the first programming language;
determining a schema associated with the first operator and a schema associated with the second operator;
determining compatibility between the schema of the first operator and the schema of the second operator; and
generating a query error message in response to incompatibility between the schema of the first operator and the schema of the second operator, wherein the generating of the query error message comprises returning the query error message to the web-based application that provided the query.

2. The computer-implemented method of claim 1 , wherein the query is provided in accordance with a pipeline command language that includes the sequence of operators in which a set of data or results produced based on execution of the first operator is applied to the second operator.

3. The computer-implemented method of claim 1 , wherein prior to identifying the sequence of operators, further comprising:
parsing the query to detect portions of the query, the portions of the query including the sequence of operators.

4. The computer-implemented method of claim 1 , wherein the schema of the first operator corresponds to an output schema of the first operator and the schema of the second operator corresponds to an input schema of the second operator.

5. The computer-implemented method of claim 4 , wherein the determining of the input schema associated with the second operator comprises accessing a data store including a machine learning model and metadata associated with the machine learning model, the metadata includes the input schema of the machine learning model.

6. The computer-implemented method of claim 5 , wherein the first operator is an executable command and the schema associated with the first operator corresponds to an output schema of the executable command.

7. The computer-implemented method of claim 1 further comprising:
permitting processing of the query in response to schemas associated with each neighboring operator of the sequence of operators being compatible to each other, including an output schema of the first operator being compatible with an input schema of the second operator.

8. The computer-implemented method of claim 1 , wherein the determining of compatibility between the schema of the first operator and the schema of the second operator comprises determining whether a data structure of an output generated by execution of the first operator by a processor provides machine data needed as input for execution of the second operator by the processor.

9. The computer-implemented method of claim 1 , wherein the determining of the schema associated with the first operator comprises accessing one or more external libraries associated with the first programing language.

10. A system comprising:
a processor; and
a non-transitory computer-readable medium having stored thereon a query validation component that, when executed by the processor, performs operations including:
obtaining a query from a web-based application in response to execution of a code cell corresponding to a plurality of statements in multiple programming languages including a first programming language and a second programming language;
identifying a sequence of operators associated with the query, the sequence of operators includes at least two neighboring operators including a first operator based on the first programming language and a second operator based on the second programming language that is different from the first programming language,
determining a schema associated with the first operator and a schema associated with the second operator,
determining compatibility between the schema of the first operator and the schema of the second operator, and
generating a query error message in response to incompatibility between the schema of the first operator and the schema of the second operator, wherein the generating of the query error message comprises returning the query error message to the web-based application that provided the query.

11. The system of claim 10 , wherein the query is provided in accordance with a pipeline command language that includes the sequence of operators in which a set of data or results produced based on execution of the first operator is applied to the second operator.

12. The system of claim 10 , wherein prior to the query validation component identifying the sequence of operators, the query validation component further performing an operation including parsing the query to detect portions of the query, the portions of the query including the sequence of operators.

13. The system of claim 10 , wherein the schema of the first operator corresponds to an output schema of the first operator and the schema of the second operator corresponds to an input schema of the second operator.

14. The system of claim 13 , wherein the query validation component determining of the input schema associated with the second operator by at least accessing a data store including a machine learning model and metadata associated with the machine learning model, the metadata includes the input schema of the machine learning model.

15. The system of claim 10 , wherein the query validation component being configured to further perform operations comprising:
permitting processing of the query in response to schemas associated with each neighboring operator of the sequence of operators being compatible to each other, including an output schema of the first operator being compatible with an input schema of the second operator.

16. The system of claim 10 , wherein the query validation component determining of compatibility between the schema of the first operator and the schema of the second operator by at least determining whether a data structure of an output generated by execution of the first operator by a processor provides machine data needed as input for execution of the second operator by the processor.

17. The system of claim 10 , wherein the query validation component determining of the schema associated with the first operator by at least accessing one or more external libraries associated with the first programing language.

18. A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause performance of operations comprising:
obtaining a query from a web-based application in response to execution of a code cell corresponding to a plurality of statements in multiple programming languages including a first programming language and a second programming language;
identifying a sequence of operators associated with the query, the sequence of operators includes at least two neighboring operators including a first operator based on the first programming language and a second operator based on the second programming language that is different from the first programming language;
determining a schema associated with the first operator and a schema associated with the second op

Assignees

• Splunk Inc

Inventors

• Kulkarni Chinmay Madhav
• Ma Lin
• Malekpour Amir
• Rajagopalan Mohan
• Reed John C
• Sriharsha Ram

Classifications

• G06F8/427

  Parsing · CPC title

• G06F16/211

  Schema design and management · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F16/953

  Querying, e.g. by the use of web search engines · CPC title

• G06F8/31 Primary

  Programming languages or programming paradigms · CPC title