Method for visualizing neural network models

Patent metadata

Field	Value
Publication number	US-10936938-B2
Application number	US-201715857587-A
Country	US
Kind code	B2
Filing date	Dec 28, 2017
Priority date	Dec 28, 2017
Publication date	Mar 2, 2021
Grant date	Mar 2, 2021

Abstract

A method for providing a graphical visualization of a neural network to a user is provided. The method includes generating the graphical visualization of the neural network at least in part by: representing layers of the neural network as respective three-dimensional blocks, wherein at least a first dimension of a given block is proportional to a computational complexity of a layer of the neural network represented by the given block; and representing data flows between the layers of the neural network as respective three-dimensional structures connecting blocks representing the layers of the neural network, wherein a first dimension of a given structure is proportional to each of a first dimension and a second dimension of a data flow represented by the given structure. The method also includes displaying the graphical visualization of the neural network to the user.

First claim

What is claimed is:

1. A method for providing a graphical visualization of a neural network to a user, the method comprising:
generating the graphical visualization of the neural network at least in part by:
representing layers of the neural network as respective three-dimensional blocks, wherein at least a first dimension of a given block is proportional to a computational complexity of a layer of the neural network represented by the given block; and
representing data flows between the layers of the neural network as respective three-dimensional structures connecting blocks representing the layers of the neural network, wherein a first dimension of a given structure is proportional to each of a first dimension and a second dimension of a data flow represented by the given structure; and

displaying the graphical visualization of the neural network to the user;
wherein the given structure uses two dimensions to represent three dimensions of the data flow represented by the given structure; and
wherein a third dimension of the given structure can be interactively manipulated by the user when the graphical visualization is displayed.

2. The method of claim 1 , wherein the first dimension of the given structure is proportional to a product of the first dimension and the second dimension of the data flow represented by the given structure.

3. The method of claim 1 , wherein a second dimension of the given structure is proportional to a third dimension of the data flow represented by the given structure.

4. The method of claim 1 , wherein the first dimension of the given structure is proportional to an area dimension of the data flow represented by the given structure, and wherein the second dimension of the given structure is proportional to a depth dimension of the data flow represented by the given structure.

5. The method of claim 1 , wherein at least a second dimension of the given block is proportional to a parameter count of the layer represented by the given block.

6. The method of claim 1 , wherein at least a second dimension of the given block is proportional to a data size of the layer represented by the given block.

7. The method of claim 1 , wherein representing the operations of the neural network as respective three-dimensional blocks, wherein at least a first dimension of a given block is proportional to a computational complexity of an operation of the neural network represented by the given block further comprises:
representing at least one of convolution and fully-connected layers of the neural network as respective three-dimensional blocks wherein a second dimension of the given block is proportional to a parameter count of the layer represented by the given block and wherein a third dimension of the given block is proportional to a data size of the layer represented by the given block; and
representing at least one of pool and softmax layer of the neural network as respective three-dimensional blocks wherein a second dimension of the given block is proportional to an area dimension of the layer represented by the given block and wherein a third dimension of the given block is proportional to a depth dimension of the layer represented by the given block.

8. The method of claim 1 , wherein the first dimension of the given block is proportional to a number of computational operations for the layer represented by the given block.

9. The method of claim 1 , wherein the first dimension of the given block is proportional to a number of floating-point operations (FLOPS) for the layer represented by the given block.

10. The method of claim 1 , wherein the first dimension of the given block is proportional to a computation time for the layer represented by the given block.

11. The method of claim 1 , wherein at least a second dimension of the given structure is proportional to a communication time for the layer represented by the given block.

12. The method of claim 1 , further comprising selecting one of a plurality of modes for the visualization, the plurality of modes comprising:
a first visualization mode, wherein the first dimension of the given block is proportional to a number of computational operations for the layer represented by the given block; and
a second visualization mode, wherein the first dimension of the given block is proportional to a computation time for the layer represented by the given block.

13. The method of claim 12 , wherein in the first visualization mode, the first dimension of the given block is proportional to a number of floating-point operations (FLOPS) for the layer represented by the given block.

14. The method of claim 12 , wherein in the second visualization mode, at least a second dimension of the given structure is proportional to a communication time for the layer represented by the given block.

15. The method of claim 12 , wherein the first visualization mode comprises a system-independent view of the neural network, and wherein the second visualization mode comprises a system-dependent view of the neural network.

16. The method of claim 1 , wherein each of the blocks within the visualization is coupled to at least a first structure representing an input data flow and at least a second structure representing an output data flow.

17. The method of claim 16 , wherein the given block is coupled to at least three structures, representing at least one of multiple input data flows and multiple output data flows for the layer represented by the given block.

18. The method of claim 1 , wherein parallel layers are represented by adjacent blocks with no structure therebetween.

19. An apparatus comprising:
a memory; and
at least one processor coupled to the memory, the processor being operative:
to generate a graphical visualization of a neural network at least in part by:
representing layers of the neural network as respective three-dimensional blocks, wherein at least a first dimension of a given block is proportional to a computational complexity of a layer of the neural network represented by the given block; and
representing data flows between the layers of the neural network as respective three-dimensional structures connecting blocks representing the layers of the neural network, wherein a first dimension of a given structure is proportional to each of a first dimension and a second dimension of a data flow represented by the given structure; and

to display the graphical visualization of the neural network to a user;
wherein the given structure uses two dimensions to represent three dimensions of the data flow represented by the given structure; and
wherein a third dimension of the given structure can be interactively manipulated by the user when the graphical visualization is displayed.

20. A computer program product comprising a non-transitory machine-readable storage medium having machine-readable program code embodied therewith, said machine-readable program code comprising machine-readable program code configured:
to generate a graphical visualization of a neural network at least in part by:
representing layers of the neural network as respective three-dimensional blocks, wherein at least a first dimension of a given block is proportional to a computational complexity of a layer of the neural network represented by the given block; and
representing data flows between the layers of the neural network as respective three-dimensional structures connecting blocks representing the layers of the neural network, wherein a first dimension of a given structur

Assignees

• IBM

Inventors

• Feng Minwei
• Ren Yufei
• Wang Yandong
• Zhang Li
• Zhang Wei

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/04 Primary

  Architecture, e.g. interconnection topology · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06F16/904 Primary

  Browsing; Visualisation therefor (for navigating the web G06F16/954; browsing optimisation for the web G06F16/957) · CPC title

• G06N3/105

  Shells for specifying net layout · CPC title