Computing apparatus using convolutional neural network and method of operating the same

Patent metadata

Field	Value
Publication number	US-11580354-B2
Application number	US-201916433484-A
Country	US
Kind code	B2
Filing date	Jun 6, 2019
Priority date	Sep 6, 2018
Publication date	Feb 14, 2023
Grant date	Feb 14, 2023

Abstract

An apparatus and a method use a convolutional neural network (CNN) including a plurality of convolution layers in the field of artificial intelligence (AI) systems and applications thereof. A computing apparatus using a CNN including a plurality of convolution layers includes a memory storing one or more instructions; and one or more processors configured to execute the one or more instructions stored in the memory to obtain input data; identify a filter for performing a convolution operation with respect to the input data, on one of the plurality of convolution layers; identify a plurality of sub-filters corresponding to different filtering regions within the filter; provide a plurality of feature maps based on the plurality of sub-filters; and obtain output data, based on the plurality of feature maps.

First claim

What is claimed is:

1. A computing apparatus using a convolutional neural network (CNN) including a plurality of convolution layers for image processing, the computing apparatus comprising:
a memory storing one or more instructions; and
one or more processors configured to execute the one or more instructions stored in the memory to:
obtain input data of an image;
identify a filter for performing, on one of the plurality of convolution layers, a convolution operation with respect to the input data;
identify a plurality of sub-filters within the filter, the plurality of sub-filters associated with a plurality of filtering regions, wherein each of the plurality of filtering regions is associated with at least one of the plurality of sub-filters;
provide a plurality of feature maps based on the plurality of sub-filters; and
obtain output data based on the plurality of feature maps, wherein the output data comprises image data corresponding to the input data;
wherein the one or more processors are further configured to execute the one or more instructions to identify the plurality of sub-filters by:
identifying a first filtering region and a second filtering region, the second filtering region being different from the first filtering region; and
identifying a selected filter associated with the first filtering region and associated with the second filtering region as one of the plurality of sub-filters,

wherein the input data comprises three-dimensional (3D) matrix data having a size of a×b×n, wherein a, b, and n are natural numbers, and
wherein the first filtering region and the second filtering region are applied for different channels among n channels, and the channels correspond to depth of the input data.

2. The computing apparatus of claim 1 , wherein the one or more processors are further configured to execute the one or more instructions to provide the plurality of feature maps by:
providing a first feature map by performing a first convolution operation with respect to the input data by using a first filter associated with the first filtering region;
providing a second feature map by performing a second convolution operation with respect to the input data by using a second filter associated with the second filtering region; and
providing a third feature map of the plurality of feature maps by performing summation with respect to the first and second feature maps.

3. The computing apparatus of claim 1 , wherein the one or more processors are further configured to execute the one or more instructions to identify the plurality of sub-filters by:
identifying the first filtering region of a predetermined size associated with the filter; and
identifying the plurality of filtering regions by shifting an association of the first filtering region in each of a first direction and a second direction with respect to the filter.

4. The computing apparatus of claim 1 , wherein the one or more processors are further configured to execute the one or more instructions to provide the plurality of feature maps by:
identifying, as one of the plurality of feature maps, a first feature map provided by performing a first convolution operation with respect to the input data, by using a first filter associated with the first filtering region;
providing a second feature map by shifting the first feature map in a first direction; and
providing a third feature map by shifting the first feature map in a second direction.

5. The computing apparatus of claim 1 , wherein
the input data corresponds to a group of a plurality of pieces of two-dimensional (2D) matrix data, and
the plurality of sub-filters are applied to a plurality of pieces of 2D matrix data, and the plurality of sub-filters are associated with different 2D filtering regions.

6. The computing apparatus of claim 1 , wherein the one or more processors are further configured to execute the one or more instructions to train the plurality of sub-filters using predetermined input data and predetermined output data.

7. The computing apparatus of claim 6 , wherein the one or more processors are further configured to execute the one or more instructions to train a first sub-filter of the plurality of sub-filters by identifying a weight of the first sub-filter and identifying a filtering region associated the first sub-filter.

8. A method of operating a convolutional neural network (CNN) including a plurality of convolution layers for image processing, the method comprising:
obtaining input data of an image;
identifying a filter for performing, on one of the plurality of convolution layers, a convolution operation with respect to the input data;
identifying a plurality of sub-filters within the filter, the plurality of sub-filters associated with a plurality of filtering regions, wherein each of the plurality of filtering regions is associated with at least one of the plurality of sub-filters;
providing a plurality of feature maps based on the plurality of sub-filters; and
obtaining output data based on the plurality of feature maps, wherein the output data comprises image data corresponding to the input data,
wherein the identifying of the plurality of sub-filters comprises:
identifying a first filtering region and a second filtering region, the second filtering region being different from the first filtering region; and
identifying a selected filter associated with the first filtering region and associated with the second filtering region as one of the plurality of sub-filters,

wherein the input data comprises three-dimensional (3D) matrix data having a size of a×b×n, wherein a, b, and n are natural numbers, and
wherein the first filtering region and the second filtering region are applied for different channels among n channels, and the channels correspond to depth of the input data.

9. The method of claim 8 , wherein the providing the plurality of feature maps further comprises:
providing a first feature map by performing a first convolution operation with respect to the input data by using a first filter associated with the first filtering region;
providing a second feature map by performing a second convolution operation with respect to the input data by using a second filter associated with the second filtering region; and
providing a third feature map of the plurality of feature maps by performing summation with respect to the first and second feature maps.

10. The method of claim 8 , wherein the identifying of the plurality of sub-filters comprises:
identifying the first filtering region of a predetermined size associated with the filter; and
identifying the plurality of filtering regions by shifting an association of the first filtering region in each of a first direction and a second direction with respect to the filter.

11. The method of claim 10 , wherein the providing the plurality of feature maps further comprises:
identifying, as one of the plurality of feature maps, a first feature map provided by performing a first convolution operation with respect to the input data, by using a first filter associated the first filtering region;
providing a second feature map by shifting the first feature map in the first direction; and
providing a third feature map by shifting the first feature map in the second direction.

12. The method of claim 8 , wherein
the input data corresponds to a group of a plurality of pieces of two-dimensional (2D) matrix data, and
the plurality of sub-filters are applied to a plurality of pieces of 2D matrix data, and the plurality of sub-filters are associated with different 2D filtering regions.

13. The method of claim 8 , further c

Assignees

• Samsung Electronics Co Ltd

Inventors

• Cho Insang
• Lee Wonjae
• Hwang Chanyoung

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/0464 Primary

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/082

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title

• G06N3/088

  Non-supervised learning, e.g. competitive learning · CPC title