Data and compute efficient equivariant convolutional networks

Patent metadata

Field	Value
Publication number	US-12412085-B2
Application number	US-202117170745-A
Country	US
Kind code	B2
Filing date	Feb 8, 2021
Priority date	Feb 6, 2020
Publication date	Sep 9, 2025
Grant date	Sep 9, 2025

Abstract

Certain aspects of the present disclosure provide a method of performing machine learning, comprising: generating a neural network model; and training the neural network model for a task with a first set of input data, wherein: the training uses a total loss function total including an equivariance loss component equivarnace according to total = task +α equivarnace , and α>0.

First claim

What is claimed is:

1. A method of machine learning, comprising:
converting an architecture of a neural network model from an equivariant architecture to a traditional convolutional architecture, wherein the traditional convolutional architecture is executable on an edge device; and
performing, at the edge device, an inference with the neural network model in the traditional convolutional architecture, wherein:
the neural network model was trained using a total loss function including a task loss component and a weighted equivariance loss component as a regularization loss component that allows the neural network model to enforce symmetries using the traditional convolutional architecture executable, and
the weighted equivariance loss component is masked based on a mask in one or more layers of the neural network model such that features in locations of an input rotated outside of a defined area are disregarded in calculating the weighted equivariance loss component.

2. The method of claim 1 , wherein:
the neural network model comprises a group equivariant neural network model comprising at least one layer ϕ mapping from an input feature space F in to an output feature space F out , and
the at least one layer ϕ guarantees that for a transformation g in a group G and a feature map f in F in , an output of the at least one layer ϕ using an input of the transformation g applied to the feature map f equals the transformation g applied to an output of the at least one layer ϕ using an input of the feature map f.

3. The method of claim 1 , wherein the neural network model comprises at least one atrous spatial pyramid pooling layer.

4. The method of claim 1 , wherein the neural network model comprises quantized weights.

5. The method of claim 4 , wherein the weights of the neural network model were quantized using cross-layer range equalization.

6. The method of claim 1 , wherein the edge device is a mobile device.

7. A processing system, comprising:
a memory comprising computer-executable instructions; and
one or more processors configured to execute the computer-executable instructions and cause the processing system to:
convert an architecture of a neural network model from an equivariant architecture to a traditional convolutional architecture, wherein the traditional convolutional architecture is executable on an edge device; and
perform, at the edge device, an inference with the neural network model in the traditional convolutional architecture, wherein:
the neural network model was trained using a total loss function including a task loss component and a weighted equivariance loss component as a regularization loss component that allows the neural network model to enforce symmetries using the traditional convolutional architecture, and
the weighted equivariance loss component is masked based on a mask in one or more layers of the neural network model such that features in locations of an input rotated outside of a defined area are disregarded in calculating the weighted equivariance loss component.

8. The processing system of claim 7 , wherein:
the neural network model comprises a group equivariant neural network model comprising at least one layer ϕ mapping from an input feature space F in to an output feature space F out , and
the at least one layer ϕ guarantees that for a transformation g in a group G and a feature map f in F in , an output of the at least one layer ϕ using an input of the transformation g applied to the feature map f equals the transformation g applied to an output of the at least one layer ϕ using an input of the feature map f.

9. The processing system of claim 7 , wherein the neural network model comprises at least one atrous spatial pyramid pooling layer.

10. The processing system of claim 7 , wherein the neural network model comprises quantized weights.

11. The processing system of claim 10 , wherein the weights of the neural network model were quantized using cross-layer range equalization.

12. The processing system of claim 7 , wherein the edge device is a mobile device.

13. A non-transitory computer-readable medium comprising computer-executable instructions that, when executed by one or more processors of a processing system, cause the processing system to perform a method, the method comprising:
converting an architecture of a neural network model from an equivariant architecture to a conventional architecture, wherein the conventional architecture is executable on an edge device; and
performing, at the edge device, an inference with the neural network model in the conventional architecture, wherein:
the neural network model was trained using a total loss function including an equivariance loss component equivarnace according to total = task +α equivarnace , where total represents a total loss, task represents a task loss component, and α represents a weight applied to the equivariance loss component;
the equivariance loss component corresponds to a regularization loss component that allows the neural network model to enforce symmetries using the conventional architecture;
the equivariance loss component is masked based on a mask in one or more layers of the neural network model such that features in locations of an input rotated outside of a defined area are disregarded in calculating the equivariance loss component; and
α>0.

14. The non-transitory computer-readable medium of claim 13 , wherein:
the neural network model comprises a group equivariant neural network model comprising at least one layer ϕ mapping from an input feature space F in to an output feature space F out , and
the at least one layer ϕ guarantees that for a transformation g in a group G and a feature map f in F in , an output of the at least one layer ϕ using an input of the transformation g applied to the feature map f equals the transformation g applied to an output of the at least one layer ϕ using an input of the feature map f.

15. The non-transitory computer-readable medium of claim 13 , wherein the neural network model comprises at least one atrous spatial pyramid pooling layer.

16. The non-transitory computer-readable medium of claim 13 , wherein the neural network model comprises quantized weights.

17. The non-transitory computer-readable medium of claim 16 , wherein the weights of the neural network model were quantized using cross-layer range equalization.

18. The non-transitory computer-readable medium of claim 13 , wherein the edge device is a mobile device.

19. A method of machine learning, comprising:
training a neural network model for a task with a first set of input data, wherein:
the training uses a total loss function including a task loss component and a weighted equivariance loss component, and
the weighted equivariance loss component is masked based on a mask in one or more layers of the neural network model such that features in locations of an input in the first set of input data rotated outside of a defined area are disregarded in calculating the weighted equivariance loss component; and

deploying the trained neural network model in an edge device.

20. The method of claim 19 , wherein:
the neural network model comprises a group equivariant neural network model comprising at least one layer ϕ mapping from an input feature space F in to an output feature space F out ,
the at least one layer ϕ guarantees that for a transformation g in a group G and a feature map f in F in , an output of the at least one layer ϕ using an input of the tran

Assignees

• Qualcomm Inc

Inventors

• Mohamed Mirgahney Husham Awadelkareem
• Cesa Gabriele
• Cohen Taco Sebastiaan
• Welling Max

Classifications

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/82

  using neural networks · CPC title