Training spectral inference neural networks using bilevel optimization

Patent metadata

Field	Value
Publication number	US-12307376-B2
Application number	US-201916972491-A
Country	US
Kind code	B2
Filing date	Jun 6, 2019
Priority date	Jun 6, 2018
Publication date	May 20, 2025
Grant date	May 20, 2025

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network having a plurality of network parameters and being configured to process an input data item to generate a feature representation comprising a values for each of a plurality of features of the input data item.

First claim

What is claimed is:

1. A method of training a neural network having a plurality of network parameters and the neural network being configured to process an input data item to generate a feature representation comprising values for each of a plurality of features of the input data item, the method comprising:
obtaining a training data set for training the neural network; and
training the neural network on the training data set to generate feature representations that approximate eigenvalues of input images or videos and that fit in memory for performing a task that operates on the feature representations of an input sequence of images or videos, wherein the sequence of images or videos are high-dimensional time series data, the training comprising:
maintaining a covariance measure of the features and a Jacobian of the covariance measure;
receiving a mini-batch comprising a plurality of pairs of training input data items, wherein each training input data item comprises a video frame of the input sequence of images or videos, respectively; and
training the neural network on the mini-batch, the training comprising:
processing each training input data item in the mini-batch using the neural network in accordance with current values of the network parameters to generate a respective feature representation of each of the training input data items;
determining, from the respective feature representations of each pair of training input data items in the mini-batch, a covariance measure update to the covariance measure;
updating, using the covariance measure update, the covariance measure;
determining, from the covariance measure update, a Jacobian update to the Jacobian of the covariance measure;
updating, using the Jacobian update, the Jacobian of the covariance measure;
determining, for each pair of training input data items, a respective kernel output;
determining, from the respective feature representations of each pair of training input data items in the mini-batch and the respective kernel output for each pair of training input data items, a kernel-weighted mini-batch covariance estimate;
determining, using the updated Jacobian of the covariance measure, the updated covariance measure, and the kernel-weighted mini-batch covariance estimate, a gradient estimate with respect to the network parameters; and
updating the current values of the network parameters using the gradient estimate.

2. The method of claim 1 , wherein the Jacobian update is a Jacobian of the covariance measure update.

3. The method of claim 1 , wherein the Jacobian update is an approximation of a Jacobian of the covariance measure update.

4. The method of claim 1 , wherein updating, using the covariance measure update, the covariance measure comprises:
interpolating between the covariance measure and the covariance measure update in accordance with a first decay rate.

5. The method of claim 4 , wherein updating, using the Jacobian update, the Jacobian of the covariance measure comprises:
interpolating between the Jacobian and the Jacobian update in accordance with a second decay rate.

6. The method of claim 5 wherein the first and second decay rate are the same.

7. The method of claim 1 , wherein determining, using the updated Jacobian of the covariance measure, the updated covariance measure, and the kernel-weighted mini-batch covariance estimate, a gradient estimate with respect to the network parameters comprises:
determining a Cholesky decomposition of the updated covariance measure;
determining, from the kernel-weighted mini-batch covariance estimate and the Cholesky decomposition, a normalized kernel-weighted covariance; and
determining the gradient estimate using the Cholesky decomposition, the normalized kernel-weighted covariance, the kernel-weighted mini-batch covariance estimate, and the updated Jacobian of the covariance measure.

8. The method of claim 7 , wherein determining, using the updated Jacobian of the covariance measure, the updated covariance measure, and the kernel-weighted mini-batch covariance estimate, a gradient estimate with respect to the network parameters comprises:
determining a Jacobian of the kernel-weighted mini-batch covariance estimate; and
determining the gradient estimate using the Cholesky decomposition, the normalized kernel-weighted covariance, the Jacobian of the kernel-weighted mini-batch covariance estimate, and the updated Jacobian of the covariance measure.

9. The method of claim 1 , wherein the input data items comprise images captured by a mechanical agent as the agent interacts with an environment, the method further comprising using the neural network to generate the feature representations for use by a reinforcement learning system controlling the agent to perform a task.

10. The method of claim 1 , further comprising, after the training of the neural network, using the neural network to perform an image or video recognition task.

11. The method of claim 1 , further comprising:
after the training, receiving a new input data item that comprises an image or a video;
processing the new input data item using the neural network to generate a feature representation of the new data item; and
storing the feature of the new data item in the memory for performing the task.

12. The method of claim 11 , further comprising:
performing the task on the new data item.

13. A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations for training a neural network having a plurality of network parameters and the neural network being configured to process an input data item to generate a feature representation comprising values for each of a plurality of features of the input data item, the operations comprising:
obtaining a training data set for training the neural network; and
training the neural network on the training data set to generate feature representations that approximate eigenvalues of input images or videos and that fit in memory for performing a task that operates on the feature representations of an input sequence of images or videos, wherein the sequence of images or videos are high-dimensional time series data, the training comprising:
maintaining a covariance measure of the features and a Jacobian of the covariance measure;
receiving a mini-batch comprising a plurality of pairs of training input data items, wherein each training input data item comprises a video frame of the input sequence of images or videos, respectively; and
training the neural network on the mini-batch, the training comprising:
processing each training input data item in the mini-batch using the neural network in accordance with current values of the network parameters to generate a respective feature representation of each of the training input data items;
determining, from the respective feature representations of each pair of training input data items in the mini-batch, a covariance measure update to the covariance measure;
updating, using the covariance measure update, the covariance measure;
determining, from the covariance measure update, a Jacobian update to the Jacobian of the covariance measure;
updating, using the Jacobian update, the Jacobian of the covariance measure;
determining, for each pair of training input data items, a respective kernel output;
determining, from the respective feature representations of each pair of training input data items in the mini-batch and the respective kernel output for each pair of training input data items, a kernel-weighted mini-batch covarianc

Assignees

• Deepmind Tech Ltd

Inventors

• Pfau David Benjamin
• Petersen Stig
• Agarwal Ashish
• Barrett David
• Stachenfeld Kimberly

Classifications

• G06N20/00

  Machine learning · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N3/006

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title

• G06N20/10

  using kernel methods, e.g. support vector machines [SVM] · CPC title

• G06N3/088 Primary

  Non-supervised learning, e.g. competitive learning · CPC title