Superpixel methods for convolutional neural networks

Patent metadata

Field	Value
Publication number	US-2018018554-A1
Application number	US-201615209658-A
Country	US
Kind code	A1
Filing date	Jul 13, 2016
Priority date	Jul 13, 2016
Publication date	Jan 18, 2018
Grant date	—

Abstract

Methods, systems, and apparatus for efficiently performing a computation of a convolutional neural network layer. One of the methods includes transforming a X by Y by Z input tensor into a X′ by Y′ by Z′ input tensor, wherein X′ is smaller than or equal to X, Y′ is smaller than or equal to Y, and Z′ is larger than or equal to Z; obtaining one or more modified weight matrices, wherein the modified weight matrices operate on the X′ by Y′ by Z′ input tensor to generate a U′ by V′ by W′ output tensor, and the U′ by V′ by W′ output tensor comprises a transformed U by V by W output tensor, wherein U′ is smaller than or equal to U, V′ is smaller than or equal to V, and W′ is larger than or equal to W; and processing the X′ by Y′ by Z′ input tensor using the modified weight matrices to generate the U′ by V′ by W′ output tensor, wherein the U′ by V′ by W′ output tensor comprises the U by V by W output tensor.

First claim

What is claimed is:

1 - 35 . (canceled)

36 . A computer implemented method for training a convolutional neural network that includes a plurality of convolutional neural network layers on training data,
wherein the convolutional neural network comprises (i) at least one superpixel convolutional neural network layer having respective superpixel convolutional neural network layer parameters that is configured to receive an X by Y by Z input tensor and process the received input tensor using one or more convolutional neural network layer weight matrices to generate an U by V by W output tensor, and (ii) one or more other convolutional neural network layers having respective layer parameters, and wherein the method comprises:
obtaining training data; and
training the convolutional neural network on the training data to adjust the values of the parameters of the superpixel convolutional neural network and the values of the parameters of the other convolutional neural network layers to trained values.

37 . The method of claim 36 , wherein receiving, by the superpixel convolutional neural network layer, an X by Y by Z input tensor and processing the received input tensor using one or more convolutional neural network layer weight matrices to generate an U by V by W output tensor comprises:
transforming the X by Y by Z input tensor into a X′ by Y′ by Z′ input tensor, wherein X′ is smaller than or equal to X, Y′ is smaller than or equal to Y, and Z′ is larger than or equal to Z; obtaining one or more modified convolutional neural network layer weight matrices, wherein (i) the modified weight matrices operate on the X′ by Y′ by Z′ input tensor to generate a U′ by V′ by W′ output tensor, and (ii) the U′ by V′ by W′ output tensor comprises a transformed U by V by W output tensor, wherein U′ is smaller than or equal to U, V′ is smaller than or equal to V, and W′ is larger than or equal to W; and processing the X′ by Y′ by Z′ input tensor using the modified weight matrices to generate the U′ by V′ by W′ output tensor, wherein the U′ by V′ by W′ output tensor comprises the U by V by W output tensor.

38 . The method of claim 36 , wherein X′ is equal to the ceiling of X divided by a natural number N, Y′ is equal to the ceiling of Y divided by a natural number M, and Z′ is equal to Z multiplied by NM.

39 . The method of claim 38 , wherein the one or more modified weight matrices comprise NM copies of each original weight corresponding to the entries of the modified weight matrices.

40 . The method of claim 39 , wherein training the convolutional neural network on the training data comprises performing a gradient descent and backpropagation neural network training process.

41 . The method of claim 40 , wherein performing the backpropagation training process comprises adding weight gradients across all NM copies to determine a net change in value for each original weight.

42 . The method of claim 38 , wherein the X by Y by Z input tensor represents XY first inputs and the X′ by Y′ by Z′ input tensor represents X′Y′ super inputs with each super input comprising a plurality of first inputs.

43 . The method of claim 42 , wherein the number of the plurality of first inputs equals NM inputs.

44 . The method of claim 38 , wherein the operation of the modified weight matrices on the X′ by Y′ by Z′ input tensor is equivalent to the operation of the superpixel convolutional neural network layer weight matrices on the X by Y by Z input tensor.

45 . The method of claim 38 , wherein entries of the modified weight matrices comprise one or more convolutional neural network layer weight matrices.

46 . The method of claim 38 , wherein the modified weight matrices comprise matrices with a banded structure.

47 . The method of claim 38 , wherein U′ is equal to the ceiling of U divided by a natural number P, V′ is equal to the ceiling of V divided by a natural number Q, and W′ is equal to W multiplied by PQ.

48 . The method of claim 47 , wherein the convolutional neural network layer includes a stride S in the X dimension and stride T in the Y dimension, and wherein the relationship between stride S, T, output tensor U by V by W and transformed output tensor U′ by V′ by W′ is given by U′=ceiling(U/NS) and V′=ceiling(V/MT).

49 . The method of claim 48 , wherein the U by V by W output tensor represents UV first outputs and the U′ by V′ by W′ output tensor represents U′V′ super outputs with each super output comprising a plurality of first outputs.

50 . The method of claim 49 , wherein the plurality of first outputs equals PQ outputs.

51 . The method of claim 37 , wherein the superpixel convolutional neural network layer comprises a pooling sub layer.

52 . The method of claim 38 , wherein the U by V by W output tensor represents UV first outputs and the U′ by V′ by W′ output tensor represents U′V′ super outputs with each super output comprising a plurality of first outputs, and
wherein the number of first outputs in the plurality of first outputs is dependent on the dimensions of the superpixel convolutional neural network layer weight matrices.

53 . The method of claim 38 , wherein the U by V by W output tensor represents UV first outputs and the U′ by V′ by W′ output tensor represents U′V′ super outputs with each super output comprising a plurality of first outputs, and
wherein the number of outputs in the plurality of first outputs is dependent on one or more of (i) the architecture of the superpixel convolutional neural network layer, (ii) an architecture of the convolutional neural network including the superpixel convolutional neural network layer, or (iii) a device implementing the superpixel convolutional neural network layer.

54 . A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations for training a convolutional neural network that includes a plurality of convolutional neural network layers on training data,
wherein the convolutional neural network comprises (i) at least one superpixel convolutional neural network layer having respective superpixel convolutional neural network layer parameters that is configured to receive an X by Y by Z input tensor and process the received input tensor using one or more convolutional neural network layer weight matrices to generate an U by V by W output tensor, and (ii) one or more other convolutional neural network layers having respective layer parameters, the operations comprising:
obtaining training data; and
training the convolutional neural network on the training data to adjust the values of the parameters of the superpixel convolutional neural network and the values of the parameters of the other convolutional neural network layers to trained values.

55 . A computer storage medium encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations for training a convolutional neural network that includes a plurality of convolutional neural network layers on training data,
wherein the convolutional neural network comprises (i) at least one superpixel convolutional neural network layer having respective superpixel convolutional neural network layer parameters that is configured to receive an X by Y by Z input tensor and process the received input tensor using one or more convolutional neural network layer weight matrices to generat

Assignees

• Google Inc

Inventors

• Young Reginald Clifford
• Ross Jonathan

Classifications

• G06N3/02 Primary

  Neural networks · CPC title

• G06N3/045 Primary

  Combinations of networks · CPC title

• G06N3/048

  Activation functions · CPC title

• G06N3/063

  using electronic means · CPC title

• G06F17/16

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title