Multi-layer neural networks using symmetric tensors

Patent metadata

Field	Value
Publication number	US-12067477-B2
Application number	US-202117526628-A
Country	US
Kind code	B2
Filing date	Nov 15, 2021
Priority date	Mar 19, 2018
Publication date	Aug 20, 2024
Grant date	Aug 20, 2024

Abstract

Methods and apparatuses for implementing a neural network using symmetric tensors. In embodiments, a system may include a higher order neural network with a plurality of layers that includes an input layer, one or more hidden layers, and an output layer. Each of the input layer, the one or more hidden layers, and the output layer includes a plurality of neurons, where the plurality of neurons includes at least first order neurons and second order neurons, and where inputs at a second order neuron are combined using a symmetric tensor.

First claim

What is claimed is:

1. A method for implementing a neural network system, the method comprising:
identifying, by the neural network system, a higher order neural network;
identifying, by the neural network system, inputs for a second order neuron of the higher order neural network;
identifying, by the neural network system, a symmetric tensor;
combining, by the neural network system, the identified inputs for the second order neuron with the identified symmetric tensor; and
outputting, by the neural network system, the combined inputs.

2. The method of claim 1 , wherein the symmetric tensor is a symmetric matrix.

3. The method of claim 1 , wherein combining the identified inputs further comprises combining, by the neural network system, a plurality of inputs and a plurality of corresponding transposed inputs.

4. The method of claim 3 , wherein the inputs are one dimensional, and the symmetric tensor is a quadratic tensor.

5. The method of claim 1 , wherein symmetric matrix represents a selected one of an ellipse, a hyperbola, a parabola, or a plane.

6. The method of claim 1 , wherein outputting the combined inputs further comprises outputting, by the neural network system, the combined inputs to an output layer.

7. The method of claim 6 , wherein the output layer is created using one quadratic perception.

8. The method of claim 1 , further comprising training, by the neural network system, the higher order neural network using a gradient descendant algorithm and an activation function.

9. The method of claim 1 , further comprising training, by the neural network system, the higher order neural network using an error function to generate a measurement of an amount of error for a given input, the amount of error representing a degree of differences of an output generated by the higher order neural network from a corresponding desired output.

10. The method of claim 1 , wherein the symmetric tensor is a cubic tensor.

11. The method of claim 1 , wherein the symmetric tensor is a quartic tensor.

12. The method of claim 2 , wherein the symmetric matrix is a 2×2 matrix.

Assignees

• Intel Corp

Inventors

• Zamora Esquivel Julio
• Cordourier Maruri Hector
• Camacho Perez Jose
• Lopez Meyer Paulo
• Cruz Vargas Jesus Adan

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/048 Primary

  Activation functions · CPC title

• G06N3/08

  Learning methods · CPC title