Deep neural network learning method and apparatus, and category-independent sub-network learning apparatus

Patent metadata

Field	Value
Publication number	US-2016110642-A1
Application number	US-201414787903-A
Country	US
Kind code	A1
Filing date	May 15, 2014
Priority date	May 23, 2013
Publication date	Apr 21, 2016
Grant date	—

Abstract

Provided is a DNN learning method that can reduce DNN learning time using data belonging to a plurality of categories. The method includes the steps of training a language-independent sub-network 120 and language-dependent sub-networks 122 and 124 with training data of Japanese and English. This step includes: a first step of training a DNN obtained by connecting neurons in an output layer of the sub-network 120 with neurons in an input layer of sub-network 122 with Japanese training data; a step of forming a DNN by connecting the sub-network 124 in place of the sub-network 122 to the sub-network 120 and training it with English data; repeating these steps alternately until all training data ends; and after completion, separating the first sub-network 120 from other sub-networks and storing it as a category-independent sub-network in a storage medium.

First claim

1 . In a deep neural network identifying objects classified to a plurality of categories, a deep neural network learning method of learning, using a computer, a category-independent sub-network used commonly for said plurality of categories, comprising:
a step where the computer stores first, second and third sub-networks in a storage medium; and a sub-network training step where the computer trains said first, second and third sub-networks with training data belonging to first and second categories among said plurality of categories; wherein said sub-network training step includes a deep neural network training step of the computer training a first deep neural network formed by connecting said second sub-network to an output side of said first sub-network with training data belonging to said first category, and training a second deep neural network formed by connecting said third sub-network to an output side of said first sub-network with training data belonging to said second category, and thereby realizing learning of said first and second deep neural networks, and a storing step of the computer separating, after completion of said deep neural network training step, said first sub-network from other sub-networks and storing it as said category-independent sub-network in a storage medium.

2 . The learning method according to claim 1 , wherein
each of said first, second and third sub-networks includes an input layer and an output layer; said deep neural network training step includes an initialization step of the computer initializing said first, second and third sub-networks, a first training step of the computer connecting neurons of said output layer of said first sub-network and neurons of said input layer of said second sub-network to form a first deep neural network, and training said first deep neural network with training data belonging to said first category, a second training step of the computer connecting neurons of said output layer of said first sub-network and neurons of said input layer of said third sub-network to form a second deep neural network, and training said second deep neural network with training data belonging to said second category, and an execution step of the computer executing said first and second training steps alternately until an end condition is satisfied.

3 . The deep neural network learning method according to claim 1 , further comprising, after completion of said sub-network training step, the step of the computer separating said second sub-network from other sub-networks and storing it as a category-dependent sub-network used for an object of said first category in a storage medium.

4 . In a deep neural network identifying objects classified to a plurality of categories, a deep neural network learning method of learning, using a computer, a category-dependent sub-network used for a specific category, comprising the steps of:
the computer storing a category-independent sub-network used commonly for said plurality of categories; the computer storing said sub-network used for a specific category; the computer initializing said sub-network used for a specific category; the computer connecting neurons of an output layer of said category-independent sub-network and neurons of an input layer of said sub-network used for a specific category and thereby forming a deep neural network; and the computer training said sub-network used for a specific category using training data belonging to said specific category while fixing parameters of said category-independent sub-network.

5 . In a deep neural network identifying objects classified to a plurality of categories, a category-independent sub-network learning apparatus for realizing learning of a category-independent sub-network used commonly for said plurality of categories, said apparatus comprising:
a storage device for storing first, second and third sub-networks; and a sub-network training device for training said first, second and third sub-networks with training data belonging to first and second categories among said plurality of categories; wherein said sub-network training apparatus includes a deep neural network training device, training a first deep neural network formed by connecting said second sub-network to an output side of said first sub-network with training data belonging to said first category, and training a second deep neural network formed by connecting said third sub-network to an output side of said first sub-network with training data belonging to said second category, and thereby realizing training of said first and second deep neural networks, and a sub-network separating device separating, after completion of training by said deep-neural network training device, said first sub-network from other sub-networks and storing it as said category-independent sub-network in a storage medium.

6 . In a deep neural network identifying objects classified to a plurality of categories, a deep neural network learning apparatus for realizing learning of a category-dependent sub-network used for a specific category, comprising:
a storage device for storing a category-independent sub-network used commonly for said plurality of categories and said sub-network used for a specific category; an initializing device for initializing said sub-network used for a specific category; a deep neural network forming device connecting neurons of an output layer of said category-independent sub-network and neurons of an input layer of said sub-network used for a specific category and thereby forming a deep neural network; and a training device training said sub-network used for a specific category using training data belonging to said specific category while fixing parameters of said category-independent sub-network.

7 . The deep neural network learning method according to claim 2 , further comprising, after completion of said sub-network training step, the step of the computer separating said second sub-network from other sub-networks and storing it as a category-dependent sub-network used for an object of said first category in a storage medium.

Assignees

• Nat Inst Inf & Comm Tech

Inventors

• Matsuda Shigeki
• Lu Xugang
• Hori Chiori
• Kashioka Hideki

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/08 Primary

  Learning methods · CPC title