Visual aids for debugging

Patent metadata

Field	Value
Publication number	US-11841789-B2
Application number	US-201816104062-A
Country	US
Kind code	B2
Filing date	Aug 16, 2018
Priority date	Jan 27, 2016
Publication date	Dec 12, 2023
Grant date	Dec 12, 2023

Abstract

An AI engine is disclosed that is configured to work with a graphical user interface (“GUI”) including, in some embodiments, one or more AI-engine modules and a visual debugging module of the GUI. A learner AI-engine module is configured to train one or more AI models on one or more concepts of a mental model defined in a pedagogical programming language. An instructor AI-engine module is configured to coordinate with one or more simulators for respectively training the one or more AI models on the mental model. The visual debugging module is configured to provide a visualization window for each AI model while the one or more AI models are at least training with the learner module respectively in the one or more simulators. A viewer can glean insight and explainability into the training of the AI models while the simulations are running and arriving at various states.

First claim

The invention claimed is:

1. A computing device, comprising:
a processor; and
a memory comprising instructions executable by the processor to
instantiate an artificial intelligence (AI) engine configured to perform one or more of training an AI model with training data received from a training source and outputting a prediction via the AI model once trained based on input data received from a data source; and
instantiate a visual debugging module configured to receive one or more of training tracking information from the AI engine while the AI model is being trained by the AI engine and prediction tracking information from the AI engine while the AI model is generating the prediction, track a progress of one or both of the training of the AI model while the AI model is being trained based on the training tracking information and the generation of the prediction via the AI model while the AI model generates the prediction based on the prediction tracking information, and output a user interface comprising a visualization window configured to visually represent the progress of one or both of the training of the AI model while the AI model is being trained, and the prediction generated via the AI model while the AI model generates the prediction, wherein the visualization window includes a graphical representation of one or more of the training tracking information and the prediction tracking information.

2. The computing device of claim 1 , wherein the visualization window comprises a graph configured to plot the training data used to train the AI model.

3. The computing device of claim 2 , wherein the graph is configured to continuously plot the training data during training.

4. The computing device of claim 3 , wherein the visualization window further comprises a pause button selectable to pause plotting of the training data.

5. The computing device of claim 2 , wherein the graph is configured to plot two or more data series, each data series selectable via a respective menu.

6. The computing device of claim 2 , wherein the instructions are executable to scroll the graph from a beginning of the training data to an end of the training data.

7. The computing device of claim 1 , wherein the visualization window further comprises a metagraph including one or more concepts of a mental model learned by the AI model.

8. The computing device of claim 7 , wherein the visualization window is configured to visually represent data going into and out of the metagraph.

9. The computing device of claim 1 , wherein the visualization window further comprises a training toggle selectable to pause the training of the AI model.

10. The computing device of claim 1 , wherein the visualization window is configured to display information regarding a calculation made during one or both of the training of the AI model and the generating of the prediction via the AI model.

11. The computing device of claim 1 , wherein the visualization window is configured to display an estimate of completion of the training of the AI model.

12. The computing device of claim 1 , wherein the visualization window is configured to visually represent which of one or more nodes of the AI model are being trained.

13. At a computing device, a method, comprising:
performing one or more of training an artificial intelligence (AI) model with training data received from a training source and outputting a prediction via the AI model once trained based on input data received from a data source;
receiving one or more of training tracking information while the AI model is being trained and prediction tracking information while the AI model is generating the prediction;
tracking a progress of one or both of training of the AI model while the AI model is being trained based on the training tracking information and generation of the prediction via the AI model while the AI model generates the prediction based on the prediction tracking information; and
outputting a user interface comprising a visualization window configured to visually represent the progress of one or both of the training of the AI model while the AI model is being trained, and the generation of the prediction via the AI model while the AI model generates the prediction, wherein the visualization window includes a graphical representation of one or more of the training tracking information and the prediction tracking information.

14. The method of claim 13 , wherein the visualization window comprises a graph configured to plot the training data used to train the AI model.

15. The method of claim 13 , wherein the visualization window further comprises a metagraph including one or more concepts of a mental model learned by the AI model.

16. The method of claim 13 , wherein the visualization window further comprises a training toggle selectable to pause the training of the AI model.

17. The method of claim 13 , wherein the visualization window is configured to display information regarding a calculation made during one or both of the training of the AI model and the determining of the prediction via the AI model.

18. The method of claim 13 , wherein the visualization window is configured to visually represent which of one or more nodes of the AI model are being trained.

19. A computing device, comprising:
a display;
a processor; and
a memory comprising instructions executable by the processor to
instantiate an artificial intelligence (AI) engine configured to perform one or more of training an AI model with training data received from a training source and outputting a prediction via the AI model once trained based on input data received from a data source; and
instantiate a visual debugging module configured to receive one or more of training tracking information from the AI engine while the AI model is being trained by the AI engine and prediction tracking information from the AI engine while the AI model is the generating the prediction, track a progress of one or both of the training of the AI model while the AI model is being trained based on the training tracking information and the generation of the prediction via the AI model while the AI model generates the prediction based on the prediction tracking information, and output, at the display, a user interface comprising a visualization window configured to visually represent the progress of one or both of the training of the AI model while the AI model is being trained, and the prediction generated via the AI model while the AI model generates the prediction, the visualization window comprising a graph configured to plot the training data used to train the AI model.

20. The computing device of claim 19 , wherein the visualization window is configured to display information regarding a calculation made during one or both of the training of the AI model and the generation of the prediction via the AI model.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Browne Keen Mcewan
• Arney Shane
• Kliman-Silver Clara Emma

Classifications

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/091

  Active learning · CPC title

• G06N3/0985

  Hyperparameter optimisation; Meta-learning; Learning-to-learn · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title