Interactive graphical user interfaces for deployment and application of neural network models using cross-device node-graph pipelines

Patent metadata

Field	Value
Publication number	US-12379836-B2
Application number	US-202217818852-A
Country	US
Kind code	B2
Filing date	Aug 10, 2022
Priority date	Aug 10, 2022
Publication date	Aug 5, 2025
Grant date	Aug 5, 2025

Abstract

A method includes providing an interactive graphical user interface comprising a first menu providing one or more input options, a second menu providing one or more machine learning models, and a third menu providing one or more output formats. The method also includes generating a graph in a portion of the interactive graphical user interface by detecting one or more user selections of an input option, a machine learning model, and an output format, displaying nodes corresponding to the input option, the machine learning model, the output format, and displaying edges connecting the first node to the second node, and the second node to the third node. The method additionally includes applying the machine learning model to an input associated with the input option to generate an output in the output format. The method further includes providing, by the interactive graphical user interface, the output in the output format.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
providing, by a computing device, an interactive graphical user interface comprising a first menu providing one or more input options, a second menu providing one or more machine learning models, and a third menu providing one or more output formats;
generating a graph in a portion of the interactive graphical user interface, wherein the generating of the graph comprises:
detecting one or more user selections of an input option from the first menu, a machine learning model from the second menu, and an output format from the third menu,
responsive to the one or more user selections, displaying, in the portion, a first node of the graph corresponding to the input option, a second node of the graph corresponding to the machine learning model, a third node of the graph corresponding to the output format, a first edge of the graph connecting the first node to the second node, and a second edge of the graph connecting the second node to the third node,
detecting another user selection of a second machine learning model from the second menu, and
responsive to the other user selection, displaying, in the portion, a fourth node of the graph corresponding to the second machine learning model, a third edge of the graph connecting the first node to the fourth node, and a fourth edge of the graph connecting the fourth node to the third node;

applying the machine learning model to an input associated with the input option to generate an output in the output format;
applying the second machine learning model to the input to generate a second output in the output format; and
providing, by the interactive graphical user interface, the output and the second output in the output format.

2. The computer-implemented method of claim 1 , further comprising:
receiving, by a second portion of the interactive graphical user interface, the input associated with the input option.

3. The computer-implemented method of claim 1 , further comprising:
receiving, by a drop-down menu linked to the third node, the output in the output format.

4. The computer-implemented method of claim 1 , further comprising:
providing, by the interactive graphical user interface, the input.

5. The computer-implemented method of claim 1 , wherein the one or more user selections comprises dragging and dropping an item from a menu into the portion.

6. The computer-implemented method of claim 1 , further comprising:
enabling a user to edit one or more parameters associated with one or more of the input, the machine learning model, or the output.

7. The computer-implemented method of claim 1 , wherein the generating of the graph further comprises:
detecting another user selection of a second output format from the third menu;
responsive to the other user selection, displaying, in the portion, a fourth node of the graph corresponding to the second output format, and a third edge of the graph connecting the second node to the fourth node; and
applying the machine learning model to the input to generate a second output in the second output format.

8. The computer-implemented method of claim 1 , wherein the other user selection comprises dragging and dropping the second machine learning model from the second menu into the portion.

9. The computer-implemented method of claim 1 , wherein the other user selection comprises uploading the second machine learning model from a library of the user.

10. The computer-implemented method of claim 1 , wherein the displaying of the first edge is responsive to a user indication connecting the first node to the second node.

11. The computer-implemented method of claim 1 , further comprises:
providing the user with a selectable edge that enables the user to confirm a connection of the first node to the second node, and
wherein the displaying of the first edge is performed upon receiving user confirmation to connect the first node to the second node.

12. The computer-implemented method of claim 1 , wherein the generating of the graph further comprises:
predicting, by a trained graph predictive model, one or more of a next node or a next edge of the graph; and
recommending the one or more of the next node or the next edge to a user.

13. The computer-implemented method of claim 1 , further comprising:
training the graph predictive model based on a plurality of graphs deployed on a plurality of computing devices.

14. The computer-implemented method of claim 1 , wherein the graph is an editable graph, and further comprising:
enabling a user to update the graph by performing one or more of adding, removing, or replacing a node, an edge, or both; and
updating the output in substantial real-time based on an update to the graph.

15. The computer-implemented method of claim 1 , wherein the providing of the output comprises providing the output to an end-user application.

16. The computer-implemented method of claim 1 , wherein the input option comprises one or more of an image, a video, an audio, or text.

17. The computer-implemented method of claim 1 , wherein the interactive graphical user interface is hosted on a platform and shared across a plurality of computing devices, and wherein one or more of the generating of the graph, the applying of the machine learning model, or the providing of the output is synchronized across the plurality of computing devices.

18. A computing device, comprising:
one or more processors; and
data storage, wherein the data storage has stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing device to carry out functions comprising:
providing, by a computing device, an interactive graphical user interface comprising a first menu providing one or more input options, a second menu providing one or more machine learning models, and a third menu providing one or more output formats;
generating a graph in a portion of the interactive graphical user interface, wherein the generating of the graph comprises:
detecting one or more user selections of an input option from the first menu, a machine learning model from the second menu, and an output format from the third menu,
responsive to the one or more user selections, displaying, in the portion, a first node of the graph corresponding to the input option, a second node of the graph corresponding to the machine learning model, a third node of the graph corresponding to the output format, a first edge of the graph connecting the first node to the second node, and a second edge of the graph connecting the second node to the third node,
detecting another user selection of a second machine learning model from the second menu, and
responsive to the other user selection, displaying, in the portion, a fourth node of the graph corresponding to the second machine learning model, a third edge of the graph connecting the first node to the fourth node, and a fourth edge of the graph connecting the fourth node to the third node;

applying the machine learning model to an input associated with the input option to generate an output in the output format;
applying the second machine learning model to the input to generate a second output in the output format; and
providing, by the interactive graphical user interface, the output and the second output in the output format.

19. An article of manufacture comprising one or more non-transitory computer readable media having computer-readable instructions stored thereon that, when executed by one

Assignees

• Google Llc

Inventors

• Du Ruofei
• Li Na
• Jin Jing
• Mandlis Maria
• Miles Scott Joseph
• Yu Ping

Classifications

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06F3/0486

  Drag-and-drop · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F3/04847 Primary

  Interaction techniques to control parameter settings, e.g. interaction with sliders or dials · CPC title