Convolutional neural networks

Patent metadata

Field	Value
Publication number	US-2016283841-A1
Application number	US-201514805704-A
Country	US
Kind code	A1
Filing date	Jul 22, 2015
Priority date	Mar 27, 2015
Publication date	Sep 29, 2016
Grant date	—

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for keyword spotting. One of the methods includes training, by a keyword detection system, a convolutional neural network for keyword detection by providing a two-dimensional set of input values to the convolutional neural network, the input values including a first dimension in time and a second dimension in frequency, and performing convolutional multiplication on the two-dimensional set of input values for a filter using a frequency stride greater than one to generate a feature map.

First claim

What is claimed is:

1 . A non-transitory computer readable storage medium storing instructions executable by a data processing apparatus and upon such execution cause the data processing apparatus to perform operations comprising:
training, by a keyword detection system, a convolutional neural network for keyword detection by:
providing a two-dimensional set of input values to the convolutional neural network, the input values including a first dimension in time and a second dimension in frequency; and
performing convolutional multiplication on the two-dimensional set of input values for a filter using a frequency stride greater than one to generate a feature map.

2 . The computer readable storage medium of claim 1 , wherein performing convolutional multiplication on the two-dimensional set of input values for the filter using the frequency stride greater than one to generate the feature map comprises performing convolutional multiplication on the two-dimensional set of input values for the filter using a frequency stride of four to generate the feature map.

3 . The computer readable storage medium of claim 1 , wherein performing convolutional multiplication on the two-dimensional set of input values for the filter using the frequency stride greater than one to generate the feature map comprises performing convolutional multiplication across the entire first dimension in time on the two-dimensional set of input values for the filter.

4 . The computer readable storage medium of claim 1 , wherein performing convolutional multiplication on the two-dimensional set of input values for the filter using a frequency stride greater than one comprises performing convolutional multiplication on the two-dimensional set of input values for the filter using a time stride of one.

5 . The computer readable storage medium of claim 1 , wherein performing convolutional multiplication on the two-dimensional set of input values for the filter using a frequency stride greater than one to generate the feature map comprises performing convolutional multiplication on the two-dimensional set of input values for the filter, the filter comprising a frequency size of eight and a time size of thirty-two.

6 . The computer readable storage medium of claim 1 , wherein performing convolutional multiplication on the two-dimensional set of input values for the filter using a frequency stride greater than one to generate the feature map comprises performing convolutional multiplication on the two-dimensional set of input values for n different filters using a frequency stride greater than one for each of the n different filters to generate n different feature maps, each of the feature maps generated using a corresponding filter.

7 . The computer readable storage medium of claim 6 , wherein performing convolutional multiplication on the two-dimensional set of input values for the n different filters using a frequency stride greater than one for each of the n different filters to generate the n different feature maps comprises performing convolutional multiplication on the two-dimensional set of input values for the n different filters, each of the n different filters having a size that is the same as the sizes of the other filters.

8 . The computer readable storage medium of claim 1 , wherein:
the convolutional neural network is a layer in a neural network that includes a second, different convolutional neural network layer, a linear low rank layer, a deep neural network layer, and a softmax layer; and training, by the keyword detection system, a convolutional neural network for keyword detection comprises:
generating, by the second, different convolutional neural network, second output using the feature map;
generating, by the linear low rank layer, a third output using the second output;
generating, by the deep neural network, a fourth output using the third output; and
generating, by the softmax layer, a final output of the neural network using the fourth output.

9 . The computer readable storage medium of claim 8 , wherein:
the feature map comprises a matrix; the second output comprises a matrix; and generating, by the linear low rank layer, a third output using the second output comprises:
creating a vector from the second output; and
generating the third output using the vector.

10 . The computer readable storage medium of claim 8 , the operations comprising:
updating the neural network using an accuracy of the final output.

11 . The computer readable storage medium of claim 1 , wherein training, by the keyword detection system, a convolutional neural network for keyword detection comprises:
updating a set of weight values for the filter using the feature map without performing a pooling operation.

12 . The computer readable storage medium of claim 11 , the operations comprising:
providing the convolutional neural network to a device for keyword detection.

13 . The computer readable storage medium of claim 11 , the operations comprising:
using the convolutional neural network for keyword detection by:
receiving an audio signal encoding an utterance;
analyzing the audio signal to identify a command included in the utterance; and
performing an action that corresponds to the command.

14 . A system comprising:
a data processing apparatus; and a non-transitory computer readable storage medium in data communication with the data processing apparatus and storing instructions executable by the data processing apparatus and upon such execution cause the data processing apparatus to perform operations comprising:
training, by a keyword detection system, a convolutional neural network for keyword detection by:
providing a two-dimensional set of input values to the convolutional neural network, the input values including a first dimension in time and a second dimension in frequency;
performing convolutional multiplication on the two-dimensional set of input values for a filter to generate a feature map; and
determining a value for a region of the feature map, the region including a time pooling dimension greater than one.

15 . The system of claim 14 , wherein determining the value for the region of the feature map, the region including a time pooling dimension greater than one comprises determining the value for the region of the feature map, the region including a time pooling dimension of two.

16 . The system of claim 14 , wherein determining the value for the region of the feature map comprises determining the value for the region, the region including a frequency pooling dimension of three.

17 . The system of claim 14 , wherein determining the value for the region of the feature map comprises determining a maximum value for the region.

18 . The system of claim 14 , wherein performing convolutional multiplication on the two-dimensional set of input values for the filter to generate the feature map comprises performing convolutional multiplication on the two-dimensional set of input values for the filter using a frequency stride of one and a time stride of one.

19 . The system of claim 14 , wherein:
the convolutional neural network is a layer in a neural network that includes a second, different convolutional neural network layer, a linear low rank layer, a deep neural network layer, and a softmax layer; and training, by the keyword detection system, a convolutional neural network for keyword detection comprises:
generating, by the second, different convolutional neural network, second output using

Assignees

• Google Inc

Inventors

• Sainath Tara N
• San Martin Maria Carolina Parada

Classifications

• G10L15/16 Primary

  using artificial neural networks · CPC title

• G10L2015/088

  Word spotting · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title