Gauge equivariant geometric graph convolutional neural network

Patent metadata

Field	Value
Publication number	US-12158922-B2
Application number	US-202117169338-A
Country	US
Kind code	B2
Filing date	Feb 5, 2021
Priority date	Feb 6, 2020
Publication date	Dec 3, 2024
Grant date	Dec 3, 2024

Abstract

Certain aspects of the present disclosure provide a method for performing machine learning, comprising: determining a plurality of vertices in a neighborhood associated with a mesh including a target vertex; determining a linear transformation configured to parallel transport signals along all edges in the mesh to the target vertex; applying the linear transformation to the plurality of vertices in the neighborhood to form a combined signal at the target vertex; determining a set of basis filters; linearly combining the basis filters using a set of learned parameters to form a gauge equivariant convolution filter, wherein the gauge equivariant convolution filter is constrained to maintain gauge equivariance; applying the gauge equivariant convolution filter to the combined signal to form an intermediate output; and applying a nonlinearity to the intermediate output to form a convolution output.

First claim

What is claimed is:

1. A method, comprising:
determining a plurality of vertices in a neighborhood associated with a mesh including a target vertex, the plurality of vertices being associated with a set of input features input into a convolutional layer of a neural network;
determining a linear transformation configured to parallel transport signals along all edges in the mesh to the target vertex;
applying the linear transformation to the plurality of vertices in the neighborhood to form a combined signal at the target vertex;
determining a set of basis filters configured to maintain gauge equivariance for each respective mesh corresponding to a respective representation sample;
linearly combining the basis filters using a set of learned parameters to form a gauge equivariant convolution filter, wherein the gauge equivariant convolution filter is constrained to maintain gauge equivariance;
transforming the combined signal to a regular representation with a plurality of representation samples, wherein each representation sample in the plurality of representation samples is rotated with respect to each other representation sample in the plurality of representation samples;
for each respective representation sample in the plurality of representation samples:
rotating the gauge equivariant convolution filter according to the rotation of the respective representation sample, and
applying the rotated gauge equivariant convolution filter to the respective representation sample;

applying the gauge equivariant convolution filter to the combined signal to form an intermediate output;
applying a gauge equivariant nonlinearity to the intermediate output to form a convolution output; and
generating, using the neural network, an inference, based on the convolution output.

2. The method of claim 1 , further comprising:
determining a non-uniform discrete Fourier transform matrix for the neighborhood; and
generating the gauge equivariant convolution filter for the neighborhood based on the non-uniform discrete Fourier transform matrix.

3. The method of claim 1 , further comprising encoding a local geometry of each respective vertex in the plurality of vertices in the neighborhood by mapping the respective vertex via a discrete equivalent of a Riemannian logarithm map in a tangent plane.

4. The method of claim 1 , wherein determining the linear transformation configured to parallel transport signals along the plurality of vertices in the mesh to the target vertex comprises aligning a tangent space at each respective vertex in the plurality of vertices, other than the target vertex, to be parallel to a tangent space of the target vertex before translating a vector between the respective vertex and the target vertex.

5. The method of claim 1 , wherein the gauge equivariant convolution filter is anisotropic.

6. The method of claim 1 , further comprising providing the convolution output to another layer of a gauge equivariant geometric graph convolutional neural network.

7. A processing system, comprising:
at least one memory comprising computer-executable instructions; and
one or more processors configured to execute the computer-executable instructions and cause the processing system to:
determine a plurality of vertices in a neighborhood associated with a mesh including a target vertex, the plurality of vertices being associated with a set of input features input into a convolutional layer of a neural network;
determine a linear transformation configured to parallel transport signals along all edges in the mesh to the target vertex;
apply the linear transformation to the plurality of vertices in the neighborhood to form a combined signal at the target vertex;
determine a set of basis filters configured to maintain gauge equivariance for each respective mesh corresponding to a respective representation sample;
linearly combine the basis filters using a set of learned parameters to form a gauge equivariant convolution filter, wherein the gauge equivariant convolution filter is constrained to maintain gauge equivariance;
transform the combined signal to a regular representation with a plurality of representation samples, wherein each representation sample in the plurality of representation samples is rotated with respect to each other representation sample in the plurality of representation samples;
for each respective representation sample in the plurality of representation samples:
rotate the gauge equivariant convolution filter according to the rotation of the respective representation sample, and
apply the rotated gauge equivariant convolution filter to the respective representation sample;

apply the gauge equivariant convolution filter to the combined signal to form an intermediate output;
apply a gauge equivariant nonlinearity to the intermediate output to form a convolution output; and
generate, using the neural network, an inference, based on the convolution output.

8. The processing system of claim 7 , wherein the one or more processors are further configured to cause the processing system to:
determine a non-uniform discrete Fourier transform matrix for the neighborhood; and
generate the gauge equivariant convolution filter for the neighborhood based on the non-uniform discrete Fourier transform matrix.

9. The processing system of claim 7 , wherein the one or more processors are further configured to cause the processing system to encode a local geometry of each respective vertex in the plurality of vertices in the neighborhood by mapping the respective vertex via a discrete equivalent of a Riemannian logarithm map in a tangent plane.

10. The processing system of claim 7 , wherein in order to determine the linear transformation configured to parallel transport signals along the plurality of vertices in the mesh to the target vertex, the one or more processors are further configured to cause the processing system to align a tangent space at each respective vertex in the plurality of vertices, other than the target vertex, to be parallel to a tangent space of the target vertex before translating a vector between the respective vertex and the target vertex.

11. The processing system of claim 7 , wherein the gauge equivariant convolution filter is anisotropic.

12. The processing system of claim 7 , wherein the one or more processors are further configured to cause the processing system to provide the convolution output to another layer of a gauge equivariant geometric graph convolutional neural network.

13. A non-transitory computer-readable medium comprising computer-executable instructions that, when executed by one or more processors of a processing system, cause the processing system to perform a method, the method comprising:
determining a plurality of vertices in a neighborhood associated with a mesh including a target vertex, the plurality of vertices being associated with a set of input features input into a convolutional layer of a neural network;
determining a linear transformation configured to parallel transport signals along all edges in the mesh to the target vertex;
applying the linear transformation to the plurality of vertices in the neighborhood to form a combined signal at the target vertex;
determining a set of basis filters configured to maintain gauge equivariance for each respective mesh corresponding to a respective representation sample;
linearly combining the basis filters using a set of learned parameters to form a gauge equivariant convolution filter, wherein the gauge equivariant convolution filter is constrained to maintain gauge equivariance;
transforming the combined signal to a regular represent

Assignees

• Qualcomm Technologies Inc

Inventors

• De Haan Pim
• Weiler Maurice
• Cohen Taco Sebastiaan
• Welling Max

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V10/426

  Graphical representations · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title