Neural architecture search with factorized hierarchical search space

Patent metadata

Field	Value
Publication number	US-11928574-B2
Application number	US-202318154321-A
Country	US
Kind code	B2
Filing date	Jan 13, 2023
Priority date	Nov 6, 2018
Publication date	Mar 12, 2024
Grant date	Mar 12, 2024

Abstract

The present disclosure is directed to an automated neural architecture search approach for designing new neural network architectures such as, for example, resource-constrained mobile CNN models. In particular, the present disclosure provides systems and methods to perform neural architecture search using a novel factorized hierarchical search space that permits layer diversity throughout the network, thereby striking the right balance between flexibility and search space size. The resulting neural architectures are able to be run relatively faster and using relatively fewer computing resources (e.g., less processing power, less memory usage, less power consumption, etc.), all while remaining competitive with or even exceeding the performance (e.g., accuracy) of current state-of-the-art mobile-optimized models.

First claim

What is claimed is:

1. A computing system, comprising:
one or more processors; and
one or more non-transitory computer-readable media that store:
a machine-learned convolutional neural network; and
instructions that, when executed by the one or more processors, cause the computing system to employ the machine-learned convolutional neural network to process input image data to output an inference;
wherein the machine-learned convolutional neural network comprises a plurality of convolutional blocks arranged in a sequence one after the other;
wherein the plurality of convolutional blocks comprise two or more convolutional blocks that each perform an inverted bottleneck convolution to produce an output; and
wherein at least two of the two or more convolutional blocks apply convolutional kernels that have different respective kernel sizes to respectively perform the inverted bottleneck convolution.

2. The computing system of claim 1 , wherein the convolutional kernels that have different respective kernel sizes to respectively perform the inverted bottleneck convolution comprise:
a 3×3 convolutional kernel applied by one of the at least two of the two or more convolutional blocks; and
a 5×5 convolutional kernel applied by another of the at least two of the two or more convolutional blocks.

3. The computing system of claim 1 , wherein the plurality of convolutional blocks comprise:
a first convolutional block configured to receive an input and perform at least one depthwise separable convolution to produce a first output;
a second convolutional block configured to receive the first output of the first convolutional block and perform at least one inverted bottleneck convolution to produce a second output;
a third convolutional block configured to receive the second output of the second convolutional block and perform at least one inverted bottleneck convolution to produce a third output;
a fourth convolutional block configured to receive the third output of the third convolutional block and perform at least one inverted bottleneck convolution to produce a fourth output;
a fifth convolutional block configured to receive the fourth output of the fourth convolutional block and perform at least one inverted bottleneck convolution to produce a fifth output;
a sixth convolutional block configured to receive the fifth output of the fifth convolutional block and perform at least one inverted bottleneck convolution to produce a sixth output; and
a seventh convolutional block configured to receive the sixth output of the sixth convolutional block and perform at least one inverted bottleneck convolution to produce a seventh output.

4. The computing system of claim 3 , wherein each of one or more layers of the first convolutional block applies a 3×3 convolutional kernel.

5. The computing system of claim 3 , wherein the first convolutional block comprises a single layer.

6. The computing system of claim 3 , wherein the first convolutional block does not include a skip connection.

7. The computing system of claim 3 , wherein each of one or more layers of the second convolutional block applies a 3×3 convolutional kernel.

8. The computing system of claim 3 , wherein the second convolutional block comprises one or both of: three identical layers, and one or more identity residual skip connections.

9. The computing system of claim 3 , wherein each of one or more layers of the third convolutional block applies a 5×5 convolutional kernel.

10. The computing system of claim 3 , wherein the third convolutional block comprises one or both of: three identical layers, and one or more identity residual skip connections.

11. The computing system of claim 3 , wherein each of one or more layers of the fourth convolutional block applies a 5×5 convolutional kernel.

12. The computing system of claim 3 , wherein the fourth convolutional block comprises one or both of: three identical layers, and one or more identity residual skip connections.

13. The computing system of claim 3 , wherein each of one or more layers of the fifth convolutional block applies a 3×3 convolutional kernel.

14. The computing system of claim 3 , wherein the fifth convolutional block comprises one or both of: two identical layers, and one or more identity residual skip connections.

15. The computing system of claim 3 , wherein each of one or more layers of the sixth convolutional block applies a 5×5 convolutional kernel.

16. The computing system of claim 3 , wherein the sixth convolutional block comprises one or both of: four identical layers, and one or more identity residual skip connections.

17. The computing system of claim 3 , wherein each of one or more layers of the seventh convolutional block applies a 3×3 convolutional kernel.

18. The computing system of claim 3 , wherein the seventh convolutional block comprises a single layer or the seventh convolutional block does not include any residual skip connections.

19. The computing system of claim 3 , wherein the machine-learned convolutional neural network further comprises:
a fully connected layer configured to receive an output of the seventh convolutional block;
a pooling layer configured to receive an output of the fully connected layer; and
an output layer configured to receive an output of the pooling layer and output the inference.

20. The computing system of claim 3 , wherein the machine-learned convolutional neural network further comprises:
an initial convolutional layer configured to receive the input image data and to perform a convolution over the image data to produce an initial output, wherein the first convolutional block is configured to receive the initial output from the initial convolutional layer.

Assignees

• Google Llc

Inventors

• Tan Mingxing
• Le Quoc
• Chen Bo
• Vasudevan Vijay
• Pang Ruoming

Classifications

• G06N3/082

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0985

  Hyperparameter optimisation; Meta-learning; Learning-to-learn · CPC title

• G06N3/09

  Supervised learning · CPC title