Visualization of a model selection process in an automated model selection system

Patent metadata

Field	Value
Publication number	US-11688111-B2
Application number	US-202016942284-A
Country	US
Kind code	B2
Filing date	Jul 29, 2020
Priority date	Jul 29, 2020
Publication date	Jun 27, 2023
Grant date	Jun 27, 2023

Abstract

Systems, computer-implemented methods, and computer program products to facilitate visualization of a model selection process are provided. According to an embodiment, a system can comprise a memory that stores computer executable components and a processor that executes the computer executable components stored in the memory. The computer executable components can comprise an interaction backend handler component that obtains one or more assessment metrics of a model pipeline candidate. The computer executable components can further comprise a visualization render component that renders a progress visualization of the model pipeline candidate based on the one or more assessment metrics.

First claim

What is claimed is:

1. A system, comprising:
a memory that stores computer executable components; and
a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise:
an interaction backend handler component that obtains one or more assessment metrics of a first model pipeline candidate and second model pipeline candidate, wherein the one or more assessment metrics comprises a percentage of training data allocated to a pipeline candidate by an automated pipeline selection process;
a visualization render component that concurrently renders a progress visualization of the first model pipeline candidate and a second progress visualization of the second model pipeline candidate based on the one or more assessment metrics, and wherein the progress visualization comprises a rendering in dots inside of other dots to indicate evaluation or training is in progress, a first type of line to indicate selected model pipeline candidates and a second type of line to indicate discarded model pipeline candidates; and
wherein the one or more assessment metrics is a build time metric, and wherein the first model pipeline candidate comprises a first combination of a machine learning model, a transformer and an estimator and wherein the second model pipeline candidate comprises a second combination of a machine learning model, a transformer and an estimator.

2. The system of claim 1 , wherein the visualization render component further renders the progress visualization in at least one of a progress map, a relationship map, or a leaderboard.

3. The system of claim 1 , wherein the progress visualization comprises a visual representation of the one or more assessment metrics.

4. The system of claim 1 , wherein the progress visualization comprises an interactive progress visualization, and wherein the visualization render component renders a tooltip visualization comprising at least one of a textual representation or a numerical representation of the one or more assessment metrics based on selection of the interactive progress visualization.

5. The system of claim 1 , wherein the visualization render component renders a ranking of the first model pipeline candidate and the second model pipeline candidate based on assessment metrics of the first model pipeline candidate and the second model pipeline candidate.

6. The system of claim 1 , wherein the computer executable components further comprise:
an action component that performs at least one of a stop operation, a save operation, or a discard operation individually corresponding to the first model pipeline candidate or the second model pipeline candidate based on a determination that a defined maximum build time has been met with respect to the first model pipeline candidate or the second model pipeline candidate that is being evaluated, thereby facilitating at least one of improved selection of a model in a model selection process, improved performance of the processor in executing the model selection process, or reduced computational costs of the processor in executing the model selection process.

7. A computer-implemented method, comprising:
obtaining, by a system operatively coupled to a processor, one or more assessment metrics of a plurality of model pipeline candidates wherein the one or more assessment metrics comprises a percentage of training data allocated to a pipeline candidate by an automated pipeline selection process;
rendering, by the system, a progress visualization of the plurality of model pipeline candidates based on the one or more assessment metrics, wherein the progress visualization comprises rendering as solid black dots to indicate that various operations have been completed and rendering in dots inside of other dots to indicate evaluation or training is in progress, a first type of line to indicate selected model pipeline candidates and a second type of line to indicate discarded model pipeline candidates; and
wherein the one or more assessment metrics is a build time metric, and wherein a first model pipeline candidate comprises a first combination of a machine learning model, a transformer and an estimator and wherein a second model pipeline candidate comprises a second combination of a machine learning model, a transformer and an estimator.

8. The computer-implemented method of claim 7 , wherein the one or more assessment metrics are selected from a group consisting of an optimization metric, a performance metric, a data allocation metric, a training data used metric, and a build time metric.

9. The computer-implemented method of claim 7 , further comprising:
rendering, by the system, the progress visualization is also at least one of a progress map, a tree based visualization, a relationship map, or a leaderboard.

10. The computer-implemented method of claim 7 , wherein the progress visualization comprises a visual representation of the one or more assessment metrics.

11. The computer-implemented method of claim 7 , wherein the progress visualization comprises an interactive progress visualization, and further comprising:
rendering, by the system, a tooltip visualization comprising at least one of a textual representation or a numerical representation of the one or more assessment metrics based on selection of the interactive progress visualization.

12. The computer-implemented method of claim 7 , further comprising:
rendering, by the system, a ranking of the plurality of model pipeline candidates based on assessment metrics of the plurality of model pipeline candidates.

13. The computer-implemented method of claim 7 , further comprising:
performing, by the system, at least one of a stop operation, a save operation, or a discard operation corresponding to only one of the plurality of model pipeline candidates based on input from an entity, thereby facilitating at least one of improved selection of a model in a model selection process, improved performance of the processor in executing the model selection process, or reduced computational costs of the processor in executing the model selection process.

14. A computer program product facilitating a visualized model selection process, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
obtain, by the processor, one or more assessment metrics of a first model pipeline candidate and a second model pipeline candidate wherein the one or more assessment metrics comprises a percentage of training data allocated to a pipeline candidate by an automated pipeline selection process;
render, by the processor, a progress visualization of the first model pipeline candidate and a progress visualization of the second model pipeline candidate based on the one or more assessment metrics, wherein the progress visualization comprises rendering in dots inside of other dots to indicate evaluation or training is in progress, a first type of line to indicate selected model pipeline candidates and a second type of line to indicate discarded model pipeline candidates; and
wherein the one or more assessment metrics is a build time metric, and wherein the first model pipeline candidate comprises a first combination of a machine learning model, a transformer and an estimator and wherein the second model pipeline candidate comprises a second combination of a machine learning model, a transformer and an estimator.

15. The computer program product of claim 14 , wherein the one or more assessment metrics are selected from

Assignees

• IBM

Inventors

• Wang Dakuo
• Chen Bei
• Yang ji hui
• Valente Abel
• Chaudhary Arunima
• Gan Chuang
• Eversman John Dillon
• Supadulya Voranouth
• Weidele Daniel Karl I
• Wang Jun
• Xu Jing James
• Patel Dhavalkumar C
• Vu Long
• Shah Syed Yousaf
• Han Si Er

Classifications

• G06T11/26 Primary

  Drawing of charts or graphs · CPC title

• G06F3/0481

  based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance · CPC title

• G06N5/00

  Computing arrangements using knowledge-based models · CPC title

• G06T11/206 Primary

  Physics · mapped topic

• G06N20/00

  Machine learning · CPC title