Techniques for determining artificial neural network topologies

Patent metadata

Field	Value
Publication number	US-11698930-B2
Application number	US-201816014495-A
Country	US
Kind code	B2
Filing date	Jun 21, 2018
Priority date	Jun 21, 2018
Publication date	Jul 11, 2023
Grant date	Jul 11, 2023

Abstract

Various embodiments are generally directed to techniques for determining artificial neural network topologies, such as by utilizing probabilistic graphical models, for instance. Some embodiments are particularly related to determining neural network topologies by bootstrapping a graph, such as a probabilistic graphical model, into a multi-graphical model, or graphical model tree. Various embodiments may include logic to determine a collection of sample sets from a dataset. In various such embodiments, each sample set may be drawn randomly for the dataset with replacement between drawings. In some embodiments, logic may partition a graph into multiple subgraph sets based on each of the sample sets. In several embodiments, the multiple subgraph sets may be scored, such as with Bayesian statistics, and selected amongst as part of determining a topology for a neural network.

First claim

The invention claimed is:

1. An apparatus, the apparatus comprising:
a memory; and
logic, at least a portion of the logic implemented in circuitry coupled to the memory, the logic to:
determine a collection of sample sets from a dataset, each sample set in the collection of sample sets drawn randomly from the dataset with replacement between drawings;
generate a graphical model tree based on the sample set collection and the dataset;
partition the graphical model tree into a plurality of subgraphs, one subgraph associated with each sample set in the collection of sample sets;
recursively partition each of the plurality of subgraphs into a plurality of levels of subgraphs;
calculate a score of each subgraph at each level of recursion, the score representing a likelihood of explaining a sample set of data by the subgraph;
at each recursive level of subgraphs, choose a subgraph with a highest score for inclusion at a next highest level of the graphical model tree and discard all other subgraphs at that level of subgraphs, until reaching a highest level of the graphical model tree; and
determine a topology for a neural network for each sample set based on the subgraph associated with that sample set.

2. The apparatus of claim 1 , the partition of each of the subgraphs comprising a descendent subgraph and two or more ancestor subgraphs.

3. The apparatus of claim 1 , the logic to recursively partition the graphical model tree into the plurality of levels of subgraphs based on conditional independence testing.

4. The apparatus of claim 1 , wherein the scores for each subgraph are calculated based on the dataset.

5. The apparatus of claim 4 , the scores for each subgraph comprising Bayesian scores.

6. The apparatus of claim 1 , the logic to generate the graphical model tree based on probabilistic graphical modeling.

7. At least one non-transitory computer-readable medium comprising a set of instructions that, in response to being executed by a processor circuit, cause the processor circuit to:
determine a collection of sample sets from a dataset, each sample set in the collection of sample sets drawn randomly from the dataset with replacement between drawings;
generate a graphical model tree based on the sample set collection and the dataset;
partition the graphical model tree into a plurality of subgraphs, one subgraph associated with each sample set in the collection of sample sets;
recursively partition each of the plurality of subgraphs into a plurality of levels of subgraphs;
calculate a score of each subgraph at each level of recursion, the score representing a likelihood of explaining a sample set of data by the subgraph;
at each recursive level of subgraphs, choose a subgraph with a highest score for inclusion at a next highest level of the graphical model tree and discard all other subgraphs at that level of subgraphs, until reaching a highest level of the graphical model tree; and
determine a topology for a neural network for each sample set based on the subgraph associated with that sample set.

8. The at least one non-transitory computer-readable medium of claim 7 , the partition of each of the subgraphs comprising a descendent subgraph and two or more ancestor subgraphs.

9. The at least one non-transitory computer-readable medium of claim 7 , comprising instructions that, in response to being executed by the processor circuit, cause the processor circuit to recursively partition the graphical model tree into the plurality of levels of subgraphs based on conditional independence testing.

10. The at least one non-transitory computer-readable medium of claim 7 , comprising instructions that, in response to being executed by the processor circuit, cause the processor circuit wherein the scores for each subgraph are calculated based on the dataset.

11. The at least one non-transitory computer-readable medium of claim 10 , the scores for each subgraph comprising Bayesian scores.

12. The at least one non-transitory computer-readable medium of claim 7 , comprising instructions that, in response to being executed by the processor circuit, cause the processor circuit to generate the graphical model tree based on probabilistic graphical modeling.

13. A computer-implemented method, comprising:
determining a collection of sample sets from a dataset, each sample set in the collection of sample sets drawn randomly from the dataset with replacement between drawings;
generating a graphical model tree based on the sample set collection and the dataset;
partitioning the graphical model tree into a plurality of subgraphs, one subgraph associated with each sample set in the collection of sample sets;
recursively partitioning each of the plurality of subgraphs into a plurality of levels of subgraphs;
calculating a score of each subgraph at each level of recursion, the score representing a likelihood of explaining a sample set of data by the subgraph;
at each recursive level of subgraphs, choosing a subgraph with a highest score for inclusion at a next highest level of the graphical model tree and discarding all other subgraphs at that level of subgraphs, until reaching a highest level of the graphical model tree; and
determining a topology for a neural network for each sample set based on the subgraph associated with that sample set.

14. The computer-implemented method of claim 13 , the partitioning of each of the subgraphs comprising a descendent subgraph and two or more ancestor subgraphs.

15. The computer-implemented method of claim 13 , comprising recursively partitioning the graphical model tree into the plurality of levels of subgraphs based on conditional independence testing.

Assignees

• Intel Corp

Inventors

• Gurwicz Yaniv
• Yehezkel Rohekar Raanan Yonatan
• Nisimov Shami
• Koren Guy
• Novik Gal

Classifications

• G06N3/045 Primary

  Combinations of networks · CPC title

• G06F16/9024 Primary

  Graphs; Linked lists (G06F16/9027 takes precedence) · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/082

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title