Efficient learning and using of topologies of neural networks in machine learning

Patent metadata

Field	Value
Publication number	US-11501152-B2
Application number	US-201715659853-A
Country	US
Kind code	B2
Filing date	Jul 26, 2017
Priority date	May 5, 2017
Publication date	Nov 15, 2022
Grant date	Nov 15, 2022

Abstract

A mechanism is described for facilitating learning and application of neural network topologies in machine learning at autonomous machines. A method of embodiments, as described herein, includes monitoring and detecting structure learning of neural networks relating to machine learning operations at a computing device having a processor, and generating a recursive generative model based on one or more topologies of one or more of the neural networks. The method may further include converting the generative model into a discriminative model.

First claim

What is claimed is:

1. An apparatus comprising:
a graphics processor to:
monitor and detect structure learning of neural networks relating to machine learning operations at the apparatus having the graphics processor, wherein the structure learning comprises learning the structure of a generative probabilistic model;
generate a recursive generative model based on the structure of the generative probabilistic model, wherein the recursive generative model is unsupervised and based on unlabeled data;
generate a stochastic inverse of the generative probabilistic model by inversing the generative probabilistic model into multiple inverse models, wherein a bidirectional connection is added to connect latent variables having a common parent in each of the multiple inverse models to consolidate the multiple inverse models into a single inverse model; and
convert the recursive generative model into a discriminative model based on the stochastic inverse of the generative probabilistic model, wherein the discriminative model is supervised and based on labeled data.

2. The apparatus of claim 1 , wherein the discriminative model is learned from the recursive generative model.

3. The apparatus of claim 1 , wherein the graphics processor is further to convert the stochastic inverse model into the discriminative model by removing the bidirectional connection and adding a class node serving as a child node to latent leaves.

4. The apparatus of claim 1 , wherein the graphics processor is further to:
perform methodological dropout of neurons from one or more of the neural networks, wherein the methodological dropout is performed in accordance with a predictivity based on historical statistical data relating to the neurons;
generate parallel and sequential execution schedules for memory sharing at sub-network precision levels of the one or more of the neural networks; and
perform on-the-fly learning and updating of network topologies of the neural networks based on at least one of currently available data and historically available data relating to the topologies of the neural networks.

5. The apparatus of claim 1 , wherein the graphics processor is further to:
facilitate at least one of an end-to-end structure learning and a sub-network structure learning; and
facilitate feature bagging or coping with large scale data by training large training sets.

6. The apparatus of claim 1 , wherein the graphics processor is co-located with an application processor on a common semiconductor package.

7. A method comprising:
monitoring and detecting structure learning of neural networks relating to machine learning operations at a graphics processor, wherein the structure learning comprises learning the structure of a generative probabilistic model;
generating a recursive generative model based on the structure of the generative probabilistic model, wherein the recursive generative model is unsupervised and based on unlabeled data;
generating a stochastic inverse of the generative probabilistic model by inversing the generative probabilistic model into multiple inverse models, wherein a bidirectional connection is added to connect latent variables having a common parent in each of the multiple inverse models to consolidate the multiple inverse models into a single inverse model; and
converting the recursive generative model into a discriminative model based on the stochastic inverse of the generative probabilistic model, wherein the discriminative model is supervised and based on labeled data.

8. The method of claim 7 , wherein the discriminative model is learned from the recursive generative model.

9. The method of claim 7 , further comprising converting the stochastic inverse model into the discriminative model by removing the bidirectional connection and adding a class node serving as a child node to latent leaves.

10. The method of claim 7 , further comprising:
performing methodological dropout of neurons from one or more of the neural networks, wherein the methodological dropout is performed in accordance with a predictivity based on historical statistical data relating to the neurons;
generating parallel and sequential execution schedules for memory sharing at sub-network precision levels of the one or more of the neural networks; and
performing on-the-fly learning and updating of network topologies of the neural networks based on at least one of currently available data and historically available data relating to the topologies of the neural networks.

11. The method of claim 7 , further comprising:
facilitating at least one of an end-to-end structure learning and a sub-network structure learning; and
facilitating feature bagging or coping with large scale data by training large training sets.

12. The method of claim 7 , wherein the graphics processor is co-located with an application processor on a common semiconductor package.

13. A non-transitory machine-readable medium comprising instructions that when executed by a computing device, cause the computing device to perform operations comprising:
monitoring and detecting structure learning of neural networks relating to machine learning operations at the computing device having a graphics processor, wherein the structure learning comprises learning the structure of a generative probabilistic model;
generating a recursive generative model based on the structure of the generative probabilistic model, wherein the recursive generative model is unsupervised and based on unlabeled data;
generating a stochastic inverse of the generative probabilistic model by inversing the generative probabilistic model into multiple inverse models, wherein a bidirectional connection is added to connect latent variables having a common parent in each of the multiple inverse models to consolidate the multiple inverse models into a single inverse model; and
converting the recursive generative model into a discriminative model based on the stochastic inverse of the generative probabilistic model, wherein the discriminative model is supervised and based on labeled data.

14. The non-transitory machine-readable medium of claim 13 , wherein the discriminative model is learned from the recursive generative model.

15. The non-transitory machine-readable medium of claim 13 , wherein the operations further comprise converting the stochastic inverse model into the discriminative model by removing the bidirectional connection and adding a class node serving as a child node to latent leaves.

16. The non-transitory machine-readable medium of claim 13 , wherein the operations further comprise:
performing methodological dropout of neurons from one or more of the neural networks, wherein the methodological dropout is performed in accordance with a predictivity based on historical statistical data relating to the neurons;
generating parallel and sequential execution schedules for memory sharing at sub-network precision levels of the one or more of the neural networks; and
performing on-the-fly learning and updating of network topologies of the neural networks based on at least one of currently available data and historically available data relating to the topologies of the neural networks.

17. The non-transitory machine-readable medium of claim 13 , wherein the operations further comprise:
facilitating at least one of an end-to-end structure learning and a sub-network structure learning; and
facilitating feature bagging or coping with large scale data by training large training sets, wherein the graphics processor is co-located with an application processor on a

Assignees

• Intel Corp

Inventors

• Yehezkel Rohekar Raanan Yonatan
• Koren Guy
• Nisimov Shami
• Novik Gal

Classifications

• G06N7/01 Primary

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06T1/20

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06N3/0445

  Physics · mapped topic