Machine learning system interface

Patent metadata

Field	Value
Publication number	US-10417577-B2
Application number	US-201514732501-A
Country	US
Kind code	B2
Filing date	Jun 5, 2015
Priority date	Jun 5, 2015
Publication date	Sep 17, 2019
Grant date	Sep 17, 2019

Abstract

Some embodiments include an experiment management interface for a machine learning system. The experiment management interface can manage one or more workflow runs related to building or testing machine learning models. The experiment management interface can receive an experiment initialization command to create a new experiment associated with a new workflow. A workflow can be represented by an interdependency graph of one or more data processing operators. The experiment management interface enables definition of the new workflow from scratch or by cloning and modifying an existing workflow. The workflow can define a summary format for its inputs and outputs. In some embodiments, the experiment management interface can automatically generate a comparative visualization at the conclusion of running the new workflow based on an input schema or an output schema of the new workflow.

First claim

What is claimed is:

1. A computer-implemented method for promoting the design and execution of machine learning processes by enabling re-use of existing workflows, the method comprising:
generating an experiment management user interface to present existing workflows that ran on a machine learning system, wherein each existing workflow is represented by an interdependency graph of one or more data processing operators, the interdependency graph defines a pipeline of the data processing operators that converts an input dataset specified by an input schema of the existing workflow into an output specified by an output schema of the existing workflow;
receiving from a user, via the experiment management user interface and with the machine learning system, an experiment initialization command to create a new experiment associated with a new workflow, wherein the experiment initialization command includes a selection of an existing workflow in the machine learning system;
receiving from the user, one or more modifications to the existing workflow via the experiment management user interface, comprising:
adding at least one data processing operator to the one or more data processing operators of the existing workflow; or
removing at least one data processing operator from the one or more data processing operators of the existing workflow;
generating, based on the received modifications, an updated pipeline of the one or more data processing operators including the added at least one data processing operator or excluding the removed at least one data processing operator, wherein the generating the updated pipeline is based on input and output schemas of the one or more data processing operators; and

causing execution of the generated new workflow by a dynamic pool of computing devices, based on the updated pipeline, by causing at least a first of the one or more data processing operators to execute on a first computing device, of the dynamic pool of computing devices, and causing at least a second of the one or more data processing operators to execute on a second computing device, of the dynamic pool of computing devices, that is different from the first computing device, and wherein causing the execution involves facilitating one or more of: load-balancing, resource consumption minimization, avoiding bottlenecks, avoiding errors, avoiding inconsistencies, or any combination thereof; and
generating a visualization to facilitate analysis of the new experiment based on an input schema or an output schema of the new workflow.

2. The computer-implemented method of claim 1 , wherein generating the experiment management user interface includes exposing the experiment management user interface usable to identify previously executed experiments on the machine learning system; and wherein receiving the experiment initialization command from the user includes receiving a user selection of a previous experiment associated with the existing workflow.

3. The computer-implemented method of claim 1 , wherein said receiving from the user the experiment initialization command includes receiving a user selection of the existing workflow at the experiment management user interface; and wherein said receiving the modifications includes, responsive to receiving the user selection, presenting a combination of at least a subset of parameters and attributes of the existing workflow for cloning or modification.

4. The computer-implemented method of claim 1 , wherein the modifications further include redefining an existing data processing operator used in the existing workflow.

5. The computer-implemented method of claim 1 , wherein the modifications further include changing an input dataset to the existing workflow.

6. The computer-implemented method of claim 1 , wherein the modifications further include changing an input data usage parameter to the existing workflow.

7. The computer-implemented method of claim 1 , wherein the modifications further include providing a definition of the added data processing operator.

8. The computer-implemented method of claim 1 , wherein the modifications include identifying a reference to a production data processing operator from a production operator library, as the added data processing operator.

9. The computer-implemented method of claim 1 , further comprising rendering an integrated development environment (IDE) text editor in the experiment management user interface to enable modification to the interdependency graph of the data processing operators, wherein the modifications are received via the IDE text editor.

10. The computer-implemented method of claim 1 , further comprising tracking a version history of the new workflow, wherein the version history identifies the existing workflow as a basis for the new workflow.

11. The computer-implemented method of claim 1 , wherein generating the visualization includes:
identifying one or more comparable experiments based on output schemas of one or more experiments tracked in an experiment repository in the machine learning system, wherein the output schemas of the comparable experiments match an output schema of the new experiment; and
responsive to a user selection of a target comparable experiment, rendering the visualization to compare the new experiment and the target comparable experiment, wherein said rendering is based on a summary format dictated by the new workflow associated with the new experiment.

12. The computer-implemented method of claim 1 , wherein generating the visualization includes:
identifying one or more comparable experiments for user selection based on a version history of the new experiment, wherein the version history tracks a provenance chain of one or more experiments that are based on one another;
receiving a user selection of a target comparable experiment from amongst the comparable experiments; and
responsive to receiving the user selection, rendering a visualization to compare the new experiment and the target comparable experiment.

13. A computer readable data memory storing computer-executable instructions that, when executed, cause a computer system to perform a computer-implemented method for promoting the design and execution of machine learning processes by enabling efficient compilation of existing data processing operators, the method, comprising:
generating an experiment management user interface to facilitate machine learning experimentations in a machine learning system;
registering an experiment with an experiment management engine of the machine learning system;
receiving from a user, via an integrated development environment of the experiment management user interface, a workflow definition text that defines a new workflow, wherein the workflow definition text describes an interdependency graph of one or more data processing operators;
compiling the new workflow by generating, based on the workflow definition text, a pipeline of the one or more data processing operators, wherein the generating the pipeline of the one or more data processing operators includes selecting at least one existing data processing operator, of the one or more data processing operators, from a pool of existing data processing operators and incorporating the existing data processing operator into the pipeline based on input and output schemas defined for the existing data processing operator:
causing execution of the generated new workflow by a dynamic pool of computing devices, based on the pipeline, by facilitating one or more of load-balancing, resource consumption minimization, avoiding bottlenecks, avoiding errors, avoiding inconsistencies, or any combination thereof;

Assignees

• Facebook Inc

Inventors

• Bowers Stuart Michael
• Mehanna Hussein Mohamed Hassan
• Gusatti Azzolini Alisson
• Dunn Jeffrey Scott
• Farnham Rodrigo Bouchardet
• Paton James Robert
• Sidorov Aleksandr
• Vagata Pamela Shen
• Xie Xiaowen

Classifications

• G06F9/453

  Help systems · CPC title

• G06F8/34

  Graphical or visual programming · CPC title

• G06N3/12

  using genetic models · CPC title

• G06N20/00 Primary

  Machine learning · CPC title