Automated artificial intelligence radial visualization

Patent metadata

Field	Value
Publication number	US-11514361-B2
Application number	US-201916557760-A
Country	US
Kind code	B2
Filing date	Aug 30, 2019
Priority date	Aug 30, 2019
Publication date	Nov 29, 2022
Grant date	Nov 29, 2022

Abstract

Embodiments for providing automated machine learning visualization. Machine learning tasks, transformers, and estimators may be received into one or more machine learning composition modules. The machine learning composition modules generate one or more machine learning models. A machine learning model pipeline is a sequence of transformers and estimators and an ensemble of machine learning pipelines are an ensemble of machine learning pipelines. A machine learning model pipeline, an ensemble of a plurality of machine learning model pipelines, or a combination thereof, along with corresponding metadata, may be generated using the machine learning composition modules. Metadata may be extracted from the machine learning model pipeline, the ensemble of a plurality of machine learning model pipelines, or combination thereof. An interactive visualization graphical user interface of the machine learning model pipeline, the ensemble of a plurality of machine learning model pipelines, or combination thereof, and the extracted metadata may be generated.

First claim

The invention claimed is:

1. A method for providing automated machine learning visualization by one or more processors, comprising:
receiving one or more machine learning tasks, one or more transformers, and one or more estimators into machine learning composition logic;
generating a machine learning model pipeline, an ensemble of a plurality of machine learning model pipelines, or a combination thereof along with corresponding metadata using the machine learning composition logic, wherein a machine learning model pipeline is a sequence of transformers and estimators and an ensemble of machine learning pipelines are an ensemble of machine learning pipelines;
extracting metadata from the machine learning model pipeline, the ensemble of a plurality of machine learning model pipelines, or combination thereof; and
generating an interactive visualization graphical user interface (“GUI”) of the machine learning model pipeline, the ensemble of a plurality of machine learning model pipelines, or combination thereof, the extracted metadata, or a combination thereof;
analyzing the extracted metadata, wherein the extracted metadata includes provenance metadata; and,
in conjunction with generating the interactive visualization GUI of the machine learning pipeline, decomposing and displaying, on the interactive visualization GUI, specific algorithms, parameters, and technical characteristics used to generate each machine learning model in the machine learning model pipeline according to the analysis of the extracted metadata and wherein the decomposing and displaying includes displaying, on the interactive visualization GUI, a list of machine learning composition structures, inclusive of specified data and specified machine learning tasks, used to generate each machine learning model in the machine learning model pipeline.

2. The method of claim 1 , wherein receiving the one or more machine learning tasks further includes receiving training data, test data, optimization metrics, evaluation metrics, target variable, the one or more transformers, and the one or more estimators into the machine learning composition logic.

3. The method of claim 1 , further including:
defining metadata for the machine learning pipeline to include structure metadata, performance metadata, the provenance metadata, or a combination thereof relating to the machine learning pipeline; or
defining metadata for the ensemble of a plurality of machine learning model pipelines to include structure metadata, performance metadata, the provenance metadata, or a combination thereof relating to the ensemble of a plurality of machine learning model pipelines.

4. The method of claim 1 , further including:
ranking the extracted metadata and the machine learning model pipeline, the ensemble of a plurality of machine learning model pipelines, or a combination thereof according to pipeline ranking criteria and metadata ranking criteria;
ranking structure metadata, performance metadata, the provenance metadata, or a combination thereof for the machine learning pipeline according to the metadata ranking criteria; or
ranking the structure metadata, the performance metadata, the provenance metadata, or a combination thereof for the ensemble of a plurality of machine learning model pipelines according to the metadata ranking criteria.

5. The method of claim 1 , further including displaying the interactive visualization GUI as a radial structure having a plurality of concentric rings having one or more nodes displayed therein, wherein the plurality of concentric rings include at least machine learning pipeline rings, data rings, estimator rings, transformer rings, and composition logic rings, wherein the one or more nodes represent the machine learning model pipeline, the ensemble of a plurality of machine learning model pipelines, or a combination thereof, data, the one or more estimators, the one or more transformers, and the machine learning composition logic used to generate the machine learning model pipeline, the ensemble of a plurality of machine learning model pipelines, or a combination thereof, wherein the having one or more nodes in the plurality of concentric rings are displayed in sequence based on different ranking criteria.

6. The method of claim 5 , further including:
associating one or more nodes with one or more of the plurality of concentric rings;
associating and displaying details relating to the machine learning model pipeline or the ensemble of a plurality of machine learning model pipelines as the user interacts with the one or more of the plurality of concentric rings; or
displaying each of the machine learning model pipeline, the ensemble of a plurality of machine learning model pipelines, or combination associated with the one or more selected data transformers, the one or more data estimators, the machine learning composition logic, one or more data partitions, or a combination thereof upon the user interacting with the one or more of the plurality of concentric rings.

7. The method of claim 5 , further including:
automatically updating the interactive visualization GUI according to one or more triggering events, or
automatically updating the interactive visualization upon a user:
selecting the pipeline ranking criteria and the metadata ranking criteria of the machine learning model pipeline, the ensemble of a plurality of machine learning model pipelines, or combination thereof the one or more transformers, the one or more estimators, the one or more machine learning composition modules, or a combination thereof for visualization in one or more corresponding rings of the interactive visualization;
selecting or interacting with one or more nodes located within the interactive visualization of the machine learning model pipeline, the ensemble of a plurality of machine learning model pipelines, or combination thereof; or
selecting or interacting with one or more nodes for of one or more of a plurality of metadata rings within the interactive visualization GUI, wherein the plurality of metadata rings include transformer rings, estimators rings, data rings, and composition modules rings.

8. A system for providing automated machine learning visualization, comprising:
one or more computers with executable instructions that when executed cause the system to:
receive one or more machine learning tasks, one or more transformers, and one or more estimators into machine learning composition logic;
generate a machine learning model pipeline, an ensemble of a plurality of machine learning model pipelines, or a combination thereof along with corresponding metadata using the machine learning composition logic, wherein a machine learning model pipeline is a sequence of transformers and estimators and an ensemble of machine learning pipelines are an ensemble of machine learning pipelines;
extract metadata from the machine learning model pipeline, the ensemble of a plurality of machine learning model pipelines, or combination thereof,
generate an interactive visualization graphical user interface (“GUI”) of the machine learning model pipeline, the ensemble of a plurality of machine learning model pipelines, or combination thereof, the extracted metadata, or a combination thereof;
analyze the extracted metadata, wherein the extracted metadata includes provenance metadata; and,
in conjunction with generating the interactive visualization GUI of the machine learning pipeline, decompose and display on the interactive visualization GUI specific algorithms, parameters, and technical characteristics used to generate each machine learning model in the machine learning model pipeline according to the analysis of the extracted metadata, and wherein the decomposing and displaying includes displayin

Assignees

• IBM

Inventors

• Salonidis Theodoros
• Eversman John
• Wang Dakuo
• Swain Alex
• Bramble Gregory
• Ju Lin
• Mazzitelli Nicholas
• Supadulya Voranouth

Classifications

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N20/20

  Ensemble learning · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06F9/542

  Event management; Broadcasting; Multicasting; Notifications · CPC title

• G06F3/04847 Primary

  Interaction techniques to control parameter settings, e.g. interaction with sliders or dials · CPC title