Image classification neural networks

Patent metadata

Field	Value
Publication number	US-12125257-B2
Application number	US-202117372090-A
Country	US
Kind code	B2
Filing date	Jul 9, 2021
Priority date	Feb 18, 2016
Publication date	Oct 22, 2024
Grant date	Oct 22, 2024

Abstract

A neural network system that includes: multiple subnetworks that includes: a first subnetwork including multiple first modules, each first module including: a pass-through convolutional layer configured to process the subnetwork input for the first subnetwork to generate a pass-through output; an average pooling stack of neural network layers that collectively processes the subnetwork input for the first subnetwork to generate an average pooling output; a first stack of convolutional neural network layers configured to collectively process the subnetwork input for the first subnetwork to generate a first stack output; a second stack of convolutional neural network layers that are configured to collectively process the subnetwork input for the first subnetwork to generate a second stack output; and a concatenation layer configured to concatenate the pass-through output, the average pooling output, the first stack output, and the second stack output to generate a first module output for the first module.

First claim

What is claimed is:

1. A system comprising one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to implement a neural network comprising:
a residual subnetwork comprising a residual module, the residual module comprising:
a first stack of convolutional neural network layers being configured to process a subnetwork input having a first dimensionality for the residual subnetwork to generate a first group output and a second stack of convolutional neural network layers being configured to process the same subnetwork input having the first dimensionality to generate a second group output, wherein each of the first group output and the second group output has a dimensionality smaller than the first dimensionality of the subnetwork input and wherein both of the first stack of convolutional neural network layers and the second stack of convolutional neural network layers are connected to a filter expansion layer;
the filter expansion layer that is configured to generate an expanded output by scaling up the dimensionality of each of the first group output and the second group output generated by processing the subnetwork input using the first stack of convolutional neural network layers and the second stack of convolutional neural network layers; and
a summing layer configured to sum (i) the same subnetwork input having the first dimensionality and (ii) a second output generated by scaling the expanded output that was generated by scaling up the dimensionality of each of the first group output and the second group output to generate a summed output.

2. The system of claim 1 , wherein the residual module further comprises a pass-through convolutional layer configured to process the subnetwork input to generate a pass-through output.

3. The system of claim 2 , wherein the filter expansion layer is configured to generate the expanded output by scaling up the dimensionality of each of the first group output, the second group output and the pass-through output.

4. The system of claim 2 , wherein the pass-through convolutional layer is a 1×1 convolutional layer.

5. The system of claim 2 , wherein the filter expansion layer is configured to receive the pass-through output, the first group output and the second group output and to apply a 1×1 convolution to the pass-through output and the first group output and the second group output to generate the expanded output.

6. The system of claim 1 , wherein the second output is the expanded output.

7. The system of claim 1 , wherein the summing layer is configured to:
scale the expanded output to generate the second output.

8. The system of claim 1 , wherein the residual module further comprises: an activation function layer configured to apply an activation function to the summed output to generate a residual module output for the residual module.

9. The system of claim 8 , wherein the activation function is a rectified linear unit (Relu) activation function.

10. The system of claim 1 , wherein the first stack of convolutional neural network layers comprises a 1×1 convolutional layer followed by a 1×1 convolutional layer.

11. The system of claim 1 , wherein the second stack of convolutional neural network layers comprises a 1×1 convolutional layer followed by a 3×3 convolutional layer followed by a 3×3 convolutional layer.

12. The system of claim 8 , wherein the residual subnetwork comprises a plurality of other residual modules and is configured to:
combine the residual module output of the residual module with other residual module outputs of other residual modules to generate a residual subnetwork output for the residual subnetwork.

13. The system of claim 1 , wherein the expanded output has a second dimensionality that matches the first dimensionality of the subnetwork input.

14. The system of claim 2 , wherein the subnetwork is a representation of an image.

15. One or more non-transitory storage media encoded with instructions that when executed by one or more computers cause the one or more computers to implement a neural network system that is configured to receive an input image and to generate a classification output for the input image, wherein the neural network system comprises:
a residual subnetwork comprising a residual module, the residual module comprising:
a first stack of convolutional neural network layers being configured to process a subnetwork input having a first dimensionality for the residual subnetwork to generate a first group output and a second stack of convolutional neural network layers being configured to process the same subnetwork input having the first dimensionality to generate a second group output, wherein each of the first group output and the second group output has a dimensionality smaller than the first dimensionality of the subnetwork input and wherein both of the first stack of convolutional neural network layers and the second stack of convolutional neural network layers are connected to a filter expansion layer;
the filter expansion layer that is configured to generate an expanded output by scaling up the dimensionality of each of the first group output and the second group output generated by processing the subnetwork input using the first stack of convolutional neural network layers and the second stack of convolutional neural network layers; and
a summing layer configured to sum (i) the same subnetwork input having the first dimensionality and (ii) a second output generated by scaling the expanded output that was generated by scaling up the dimensionality of each of the first group output and the second group output to generate a summed output.

16. The one or more non-transitory storage media of claim 15 , wherein the residual module further comprises a pass-through convolutional layer configured to process the subnetwork input to generate a pass-through output.

17. The one or more non-transitory storage media of claim 16 , wherein the filter expansion layer is configured to generate the expanded output by scaling up the dimensionality of each of the first group output, the second group output and the pass-through output.

18. The one or more non-transitory storage media of claim 15 , wherein the residual module further comprises: an activation function layer configured to apply an activation function to the summed output to generate a residual module output for the residual module.

19. The one or more non-transitory storage media of claim 18 , wherein the residual subnetwork comprises a plurality of other residual modules and is configured to:
combine the residual module output of the residual module with other residual module outputs of other residual modules to generate a residual subnetwork output for the residual subnetwork.

20. The one or more non-transitory storage media of claim 15 , wherein the subnetwork is a representation of an image.

Assignees

• Google Llc

Inventors

• Vanhoucke Vincent O
• Szegedy Christian
• Ioffe Sergey

Classifications

• G06N3/08

  Learning methods · CPC title

• G06N3/045 Primary

  Combinations of networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F18/24

  Classification techniques · CPC title