Scheduling a notebook execution

Patent metadata

Field	Value
Publication number	US-9990230-B1
Application number	US-201615052204-A
Country	US
Kind code	B1
Filing date	Feb 24, 2016
Priority date	Feb 24, 2016
Publication date	Jun 5, 2018
Grant date	Jun 5, 2018

Abstract

A system for scheduling a notebook execution includes an interface and a processor. The interface is to receive an indication to schedule a notebook for execution, wherein the indication comprises a scheduled time and a cluster. The processor is to determine whether it is the scheduled time; and in the event that it is the scheduled time: determine whether the cluster is running; and in the event that the cluster is not running, set up the cluster and cause the notebook to execute using the cluster.

First claim

What is claimed is:

1. A system for scheduling a notebook execution, comprising:
an interface to receive an indication to schedule a plurality of notebooks for execution, wherein each of the plurality of notebooks includes a sequential interactive code file with at least one code section and at least one version, and the indication comprises: a scheduled time, a cluster, and a version of the notebook to be executed; and
a physical processor to:
determine that it is the scheduled time; and
in response to the determination that it is the scheduled time:
determine that the cluster is not running;
in response to the determination that the cluster is not running, set up a cluster instantiation for each of the plurality of notebooks to be executed including providing cluster configuration information to the instantiated cluster, the cluster configuration information including a type of operating system; and
instruct the instantiated cluster to execute at least one of the plurality of notebooks.

2. The system of claim 1 , wherein the indication comprises a notebook name.

3. The system of claim 1 , wherein the indication comprises input parameters.

4. The system of claim 1 , wherein the indication comprises cluster configuration information.

5. The system of claim 1 , wherein the indication comprises authentication information.

6. The system of claim 1 , wherein the indication comprises an output storage location.

7. The system of claim 1 , wherein the indication comprises a notification method.

8. The system of claim 1 , wherein the cluster comprises a group of subsystems.

9. The system of claim 1 , wherein setting up the cluster instantiation comprises providing an indication to the cluster to start.

10. The system of claim 1 , wherein instructing the instantiated cluster to execute at least one of the plurality of notebooks comprises providing notebook information to the cluster.

11. The system of claim 1 , wherein instructing the instantiated cluster to execute at least one of the plurality of notebooks comprises providing authentication information to the cluster.

12. The system of claim 1 , wherein instructing the instantiated cluster to execute at least one of the plurality of notebooks comprises providing an output storage location to the cluster.

13. The system of claim 1 , wherein the physical processor is further to access output data stored in an output storage location.

14. The system of claim 1 , wherein the scheduled time comprises a specific scheduled time.

15. The system of claim 1 , wherein the scheduled time comprises a recurring scheduled time.

16. The system of claim 1 , wherein the physical processor is further configured to, in response to a determination that the cluster is not running, determine whether the cluster is running after a pre-defined delay time.

17. The system of claim 1 , wherein the set up of the cluster instantiation includes providing cluster configuration information to the cluster, the cluster configuration information including hardware specifications of computers in the cluster.

18. A method for scheduling a notebook execution, comprising:
receiving an indication to schedule a plurality of notebooks for execution, wherein each of the plurality of notebooks includes a sequential interactive code file with at least one code section and at least one version, and the indication comprises: a scheduled time, a cluster, and a version of the notebook to be executed;
determining, using a physical processor, that it is the scheduled time; and
in response to the determination that it is the scheduled time:
determining that the cluster is not running;
in response to the determination that the cluster is not running, setting up a cluster instantiation for each of the plurality of notebooks to be executed including providing cluster configuration information to the instantiated cluster, the cluster configuration information including a type of operating system; and
instructing the instantiated cluster to execute at least one of the plurality of notebooks.

19. A computer program product for scheduling a notebook execution, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
receiving an indication to schedule a plurality of notebooks for execution, wherein each of the plurality of notebooks includes a sequential interactive code file with at least one code section and at least one version, and the indication comprises: a scheduled time, a cluster, and a version of the notebook to be executed;
determining that it is the scheduled time; and
in response to the determination that it is the scheduled time:
determining that the cluster is not running;
in response to the determination that the cluster is not running, setting up a cluster instantiation for each of the plurality of notebooks to be executed including providing cluster configuration information to the instantiated cluster, the cluster configuration information including a type of operating system; and
instructing the instantiated cluster to execute at least one of the plurality of notebooks.

Assignees

• Databricks Inc

Inventors

• Stoica Ion
• Mao Yandong
• Liang Eric

Classifications

• G06F9/4887 Primary

  involving deadlines, e.g. rate based, periodic · CPC title

• G06F9/4843 Primary

  by program, e.g. task dispatcher, supervisor, operating system · CPC title