Extracting sequences from d-dimensional input data for sequential processing with neural networks

Patent metadata

Field	Value
Publication number	US-12165028-B2
Application number	US-202016940925-A
Country	US
Kind code	B2
Filing date	Jul 28, 2020
Priority date	Jul 28, 2020
Publication date	Dec 10, 2024
Grant date	Dec 10, 2024

Abstract

A method and computer program product for obtaining values are run using a neural network according to a machine learning algorithm. One embodiment may comprise accessing one or more datafiles of input data, where the input data is representable in a d-dimensional space, with d>1. The method may explore N distinct paths of the input data in the d-dimensional space, where N≥1, and collects data along the N distinct paths explored to respectively form N sequences of M objects each, with M≥2. For one or more sequences of the N sequences formed, values obtained from the M objects of each sequence may be coupled into one or more input nodes of a neural network, which is then run according to the machine learning algorithm to obtain L output values from, L≥1.

First claim

What is claimed is:

1. A computer-implemented method of obtaining values by running a neural network according to a machine learning algorithm designed to operate on sequences of data, the method comprising:
accessing one or more datafiles of input data representable in a d-dimensional space, wherein d>1;
by a processor, forming N sequences of M objects each by exploring N distinct paths of said input data in the d-dimensional space and collecting data along the N distinct paths explored, wherein N≥1 and M≥2;
for one or more sequences of the N sequences formed, by the processor, coupling values obtained from the M objects of each sequence into one or more input nodes of a neural network and running the neural network according to said machine learning algorithm to obtain L output values for each sequence of the N sequences, wherein L≥1, and wherein N sets of L output values are obtained for the N sequences;
by the processor, training parameters of a cognitive model using the N sets of L output values; and
by the processor, performing N inferences for the N sequences using the cognitive model after training.

2. The computer-implemented method according to claim 1 , further comprising:
prior to coupling said values, setting, for each of the N sequences formed, the network according to a given set of parameters of a cognitive model.

3. The computer-implemented method according to claim 2 , further comprising:
transmitting the prediction or the classification based on the N inferences performed.

4. The computer-implemented method according to claim 1 , wherein:
N>1 and the method further comprises averaging out outcomes from said N inferences.

5. The computer-implemented method according to claim 1 , wherein:
said machine learning algorithm is a cognitive algorithm and the network is run according to said cognitive algorithm to train a cognitive model in a supervised manner with respect to a same set of L target output values for each of the N sequences formed, permitting the cognitive algorithm to learn N sets of parameters of the cognitive model based on the same set of L target output values.

6. The computer-implemented method according to claim 5 , wherein:
N>1 and the method further comprises obtaining an ensemble model based on the N sets of parameters learned.

7. The computer-implemented method according to claim 6 , wherein the method further comprises, after having obtained the ensemble model, performing inferences by:
accessing one or more further datafiles of further input data representable in the d-dimensional space;
exploring N′ distinct paths of said further input data in the d-dimensional space, wherein the N′ distinct paths are selected from the N distinct paths previously explored, 1≤N′≤N, and, based on the further input data, collecting data along the N′ distinct paths explored to respectively form N′ sequences of M objects each; and
for each sequence of the N′ sequences formed, coupling values obtained from the M objects of each of the N′ sequences formed into the one or more input nodes of the neural network and running the latter to obtain L output values, to eventually obtain N′ sets of L output values.

8. The computer-implemented method according to claim 7 , wherein:
the cognitive model obtained includes a set of N base learners, each corresponding to a respective one of the N distinct paths; and
the method further comprises selecting N′ base learners corresponding to respective ones of the N′ distinct paths selected and assigning each of the N′ base learners selected to a respective one of the N′ sequences formed.

9. The computer-implemented method according to claim 5 , wherein the method further comprises, while the network is being run to train the cognitive model, and for each sequence of one or more of the N sequences formed:
pausing the cognitive algorithm that is being trained; and
coupling values obtained from the M objects of distinct one of the N sequences formed into the one or more input nodes of the network and resuming the training of the cognitive algorithm.

10. The computer-implemented method according to claim 1 , further comprising:
devising each path of said N distinct data paths according to a random walk.

11. The computer-implemented method according to claim 10 , wherein:
at devising said each path, step sizes of said random walk are chosen at random, in addition to said directions.

12. The computer-implemented method according to claim 10 , wherein:
devising said random walk comprises, at each step j thereof, j≥2, generating a j th direction at random in said d-dimensional space, merging the j th direction generated with the j−1 th direction as generated at step j−1 of the walk to obtain a merged direction, and advancing the walk by one step in the merged direction.

13. The computer-implemented method according to claim 1 , further comprising:
devising each path of said N distinct path according to a random shot.

14. The computer-implemented method according to claim 1 , wherein:
the network is run according to a recurrent neural network algorithm.

15. The computer-implemented method according to claim 1 , wherein:
the network is run according to an echo state network algorithm.

16. The computer-implemented method according to claim 1 , wherein:
the network is run according to a feedforward neural network algorithm.

17. The computer-implemented method according to claim 1 , wherein:
each of the M objects of each of the N sequences is a k-dimensional object, k≥1, comprising N k values, N k ≥2.

18. The computer-implemented method according to claim 17 , wherein:
the network comprises N k input nodes; and
the method further comprises, for each of the N sequences, sequentially coupling M sets of values into said N k input nodes, in order to sequentially run the network for each of M objects of each of the N sequences, according to a sequential processing algorithm.

19. The computer-implemented method according to claim 17 , wherein:
k≥2.

20. A computer program product for obtaining values by running a neural network according to a machine learning algorithm designed to operate on sequences of data, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor, so as to cause the processor to:
access one or more datafiles of input data representable in a d-dimensional space, d>1;
explore N distinct paths of said input data in the d-dimensional space, N≥1, and collect data along the N distinct paths explored to respectively form N sequences of M objects each, M≥2;
for each sequence of the N sequences formed, couple values obtained from the M objects of said each sequence into one or more input nodes of a neural network and run the neural network according to said machine learning algorithm to obtain L output values for each sequence of the N sequences, L≥1, and wherein N sets of L output values are obtained for the N sequences;
by the processor, training parameters of a cognitive model using the N sets of L output values; and
by the processor, performing N inferences for the N sequences using the cognitive model after training.

Assignees

• IBM

Inventors

• Muller Lorenz K
• Stark Pascal
• Abel Stefan

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/049

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title

• G06N3/08

  Learning methods · CPC title

• G06V10/82 Primary

  using neural networks · CPC title