Data processing apparatus, training apparatus, method of detecting an object, method of training, and medium

Patent metadata

Field	Value
Publication number	US-12079717-B2
Application number	US-202016918206-A
Country	US
Kind code	B2
Filing date	Jul 1, 2020
Priority date	Jul 2, 2019
Publication date	Sep 3, 2024
Grant date	Sep 3, 2024

Abstract

There is provided with a data processing apparatus for detecting an object from an image using a hierarchical neural network. The data processing apparatus has parallel first and second neural networks. An obtaining unit obtains a table which defines different first and second portions. An operation unit performs calculation of the feature data of a third portion based on feature data of the first portion identified using the table and on a weighting parameter between first and second layers of the first neural network, and calculation of feature data of a fourth portion based on feature data of the second portion identified using the table and on a weighting parameter between the first and second layers of the second neural network.

First claim

What is claimed is:

1. A data processing apparatus for detecting an object from an image using a hierarchical neural network, comprising:
at least one memory storing computer-executable instructions and at least one processor configured to execute the computer-executable instructions stored in the at least one memory and/or at least one circuit;
parallel first and second neural networks;
an obtaining unit configured to obtain a table which defines different first and second portions, wherein the first portion comprises both a first part of channels of a first layer of the first neural network and a first part of channels of a first layer of the second neural network, both of the first parts of channels being referenced in order to obtain feature data of a third portion of a second layer of the first neural network, wherein the second portion comprises both a second part of channels of the first layer of the first neural network, different from the first part of channels of the first layer of the first neural network, and a second part of channels of the first layer of the second neural network, different from the first part of channels of the first layer of the second neural network, both of the second parts of channels being referenced, without both of the first parts of channels being referenced, in order to obtain feature data of a fourth portion of a second layer of the second neural network, and wherein both of the first parts of channels are referenced, without both of the second parts of channels being referenced, in order to obtain the feature data of the third portion of the second layer of the first neural network; and
an operation unit configured to obtain the feature data of the third portion based on feature data of the first portion identified using the table and on a weighting parameter between the first and second layers of the first neural network, and to obtain the feature data of the fourth portion based on feature data of the second portion identified using the table and on a weighting parameter between the first and second layers of the second neural network,
wherein the at least one memory storing computer-executable instructions and the at least one processor configured to execute the computer-executable instructions stored in the at least one memory and/or the at least one circuit are configured to implement at least the obtaining unit and the operation unit.

2. The data processing apparatus according to claim 1 , wherein the first portion and the second portion of the first layer do not overlap.

3. The data processing apparatus according to claim 1 , wherein the first portion and the second portion of the first layer partially overlap.

4. The data processing apparatus according to claim 1 , wherein a size of the third portion of the second layer is different from a size of the fourth portion of the second layer.

5. The data processing apparatus according to claim 1 , wherein the at least one memory storing computer-executable instructions and the at least one processor configured to execute the computer-executable instructions stored in the at least one memory and/or the at least one circuit are further configured to implement an instruction unit configured to instruct an object type,
wherein the obtaining unit is further configured to obtain the table corresponding to the instructed object type.

6. The data processing apparatus according to claim 1 , wherein
the operation unit comprises a first processing unit and a second processing unit which operate in parallel,
the first processing unit is configured to obtain the feature data of the third portion of the second layer from the feature data of the first portion of the first layer, and
the second processing unit is configured to obtain feature data of the fourth portion of the second layer from the feature data of the second portion of the first layer.

7. The data processing apparatus according to claim 1 , wherein a connection parameter is stored in the table, and the connection parameter defines the first portion and the second portion of the first layer in units of channels.

8. The data processing apparatus according to claim 7 , wherein the connection parameter defines the first portion and the second portion of the first layer in units of blocks that each includes a plurality of channels.

9. The data processing apparatus according to claim 7 , wherein the operation unit is further configured to obtain the feature data of each channel included in the third portion of the second layer using the feature data of all channels included in the first portion of the first layer and without using the feature data of the channels included in the second portion of the first layer.

10. The data processing apparatus according to claim 1 , wherein the neural network is a convolutional neural network or a recursive neural network.

11. A training apparatus operable to perform training of a hierarchical neural network, the apparatus comprising:
at least one memory storing computer-executable instructions and at least one processor configured to execute the computer-executable instructions stored in the at least one memory and/or at least one circuit configured to implement at least:
an obtaining unit configured to obtain training data and supervisory data indicating processing results for the training data;
a data processing unit configured to obtain a result of processing the training data by inputting the training data to the neural network, the data processing unit comprising an operation unit configured to, in accordance with a connection parameter defining a first portion of a first layer of the neural network to be referenced in order to obtain feature data of a third portion of a second layer of the neural network and defining a second portion of the first layer to be referenced in order to obtain feature data of a fourth portion of the second layer, obtain the feature data of the third portion of the second layer from feature data of the first portion of the first layer and obtain the feature data of the fourth portion of the second layer from feature data of the second portion of the first layer; and
a training unit configured to perform training of the connection parameter and weighting coefficients between layers of the neural network based on the supervisory data and a result of processing the training data,
wherein the neural network comprises a first neural network and a second neural network, and
wherein the first portion comprises both a first part of channels of a first layer of the first neural network and a first part of channels of a first layer of the second neural network, both of the first parts of channels being referenced in order to obtain feature data of a third portion of a second layer of the first neural network, wherein the second portion comprises both a second part of channels of the first layer of the first neural network, different from the first part of channels of the first layer of the first neural network, and a second part of channels of the first layer of the second neural network, different from the first part of channels of the first layer of the second neural network, both of the second parts of channels being referenced, without both of the first parts of channels being referenced, in order to obtain feature data of a fourth portion of a second layer of the second neural network, and wherein both of the first parts of channels are referenced, without both of the second parts of channels being referenced, in order to obtain the feature data of the third portion of the second layer of the first neural network.

12. The training apparatus according to claim 11 , wherein t

Assignees

• Canon Kk

Inventors

• Tate Shunta
• Chen Tsewei

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title