Optimizer based prunner for neural networks

Patent metadata

Field	Value
Publication number	US-11580400-B1
Application number	US-201916586635-A
Country	US
Kind code	B1
Filing date	Sep 27, 2019
Priority date	Sep 27, 2019
Publication date	Feb 14, 2023
Grant date	Feb 14, 2023

Abstract

A neural network pruning system can sparsely prune neural network models using an optimizer based approach that is agnostic to the model architecture being pruned. The neural network pruning system can prune by operating on the parameter vector of the full model and the gradient vector of the loss function with respect to the model parameters. The neural network pruning system can iteratively update parameters based on the gradients, while zeroing out as many parameters as possible based a preconfigured penalty.

First claim

What is claimed is:

1. A method comprising:
receiving, at a client device, a sparsely pruned generative neural network model from a server that is configured to:
train a full generative neural network model, and
generate the sparsely pruned generative neural network model by: applying a sparse pruning optimizer that is agnostic to the full generative neural network model, and individually pruning parameters of the full generative neural network model;

generating an image on a client device;
generating, on the client device, image output data by applying the sparsely pruned generative neural network model to the image; and
storing the image output data on the client device;
generating a modified image based on the image output data; and
publishing the modified image as an ephemeral message on a social network site.

2. The method of claim 1 , wherein the sparsely pruned generative neural network model is generated by pruning non-zero parameters in a parameter vector of the full generative neural network model based on gradient data in a gradient vector of the full generative neural network model.

3. The method of claim 2 , wherein individually pruning parameters comprises zeroing non-zero parameters based on corresponding gradient data.

4. The method of claim 1 , wherein the image output data is an image mask generated at least in part by segmentation of the image.

5. The method of claim 4 , further comprising:
generating the modified image using the image mask; and
storing the modified image on the client device.

6. The method of claim 5 , further comprising:
transmitting the modified image to another client device over a network.

7. The method of claim 1 , wherein individually pruning parameters comprises zeroing non-zero parameters based on corresponding gradient data.

8. The method of claim 1 , wherein generating the image output data further comprises:
applying the sparsely pruned generative neural network model to the image to generate image mask data labelling a region depicted in the image; and
applying an image effect to the region depicted in the image to generate the modified image.

9. The method of claim 8 , wherein the modified image is part of a modified video sequence.

10. The method of claim 1 , wherein publishing the modified image comprises sending the modified image to the social network site.

11. A client device comprising:
one or more processors of a machine; and
a memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising:
receiving, at a client device, a sparsely pruned generative neural network model from a server that is configured to:
train a full generative neural network model, and
generate the sparsely pruned generative neural network model by: applying a sparse pruning optimizer that is agnostic to the full generative neural network model, and individually pruning parameters of the full generative neural network model;

generating an image on a client device;
generating, on the client device, image output data by applying the sparsely pruned generative neural network model to the image;
storing the image output data on the client device;
generating a modified image based on the image output data; and
publishing the modified image as an ephemeral message on a social network site.

12. The client device of claim 11 , wherein the sparsely pruned generative neural network is generated by pruning non-zero parameters in a parameter vector of the full generative neural network model based on gradient data in a gradient vector of the full generative neural network model.

13. The client device of claim 12 , wherein individually pruning parameters comprises zeroing non-zero parameters based on corresponding gradient data.

14. The client device of claim 11 , wherein the image output data is an image mask generated at least in part by segmentation of the image.

15. The client device of claim 14 , the operations further comprising:
generating the modified image using the image mask; and
storing the modified image on the client device.

16. The client device of claim 15 , the operations further comprising:
transmitting the modified image to another client device over a network.

17. The client device of claim 11 , wherein individually pruning parameters comprises zeroing non-zero parameters based on corresponding gradient data.

18. A machine-readable storage device embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:
receiving, at a client device, a sparsely pruned generative neural network model from a server that is configured to:
train a full generative neural network model, and
generate the sparsely pruned generative neural network model by: applying a sparse pruning optimizer that is agnostic to the full generative neural network model, and individually pruning parameters of the full generative neural network model;

generating an image on the client device;
generating, on the client device, image output data by applying the sparsely pruned generative neural network model to the image; and
storing the image output data on the client device;
generating a modified image based on the image output data; and
publishing the modified image as an ephemeral message on a social network site.

19. The machine-readable storage device of claim 18 , wherein the sparsely pruned generative neural network model is generated by pruning non-zero parameters in a parameter vector of the full generative neural network model based on gradient data in a gradient vector of the full generative neural network model.

20. The machine-readable storage device of claim 19 , wherein individually pruning parameters comprises zeroing non-zero parameters based on corresponding gradient data.

Assignees

• Snap Inc

Inventors

• Yan Enxu
• Tulyakov Sergey
• Podkin Aleksei
• Stoliar Aleksei

Classifications

• G06V10/82

  using neural networks · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title