Image processing method and device, computer storage medium and terminal

Patent metadata

Field	Value
Publication number	US-12243188-B2
Application number	US-202017618534-A
Country	US
Kind code	B2
Filing date	Apr 16, 2020
Priority date	Jun 14, 2019
Publication date	Mar 4, 2025
Grant date	Mar 4, 2025

Abstract

The present application provides an image processing method, an image processing device, a computer storage medium and a terminal, the image processing method includes: determining convolution kernels of at least two sizes for feature extraction; performing sparsity constraint for the determined convolution kernels of at least two sizes for feature extraction through a preset objective function; and performing feature extraction on an image based on the convolution kernels subjected to the sparsity constraint.

First claim

What is claimed is:

1. An image processing method, comprising:
determining convolution kernels of at least two sizes for feature extraction;
performing sparsity constraint for the determined convolution kernels of at least two sizes for feature extraction through a preset objective function; and
performing feature extraction on one convolutional layer of an image based on the convolution kernels of at least two sizes subjected to the sparsity constraint, wherein at least two sizes of the convolution kernels for feature extraction comprise at least two of following sizes: 3×3, 5×5, 7×7, or 9×9,
wherein the determining the convolution kernels of at least two sizes for feature extraction comprises:
determining at least two sizes of the convolution kernels for feature extraction according to a preset strategy of size selection;
determining a number of convolution kernels of each size of the at least two sizes according to a preset allocation strategy after the at least two sizes of the convolution kernels for feature extraction are determined,
wherein the determining the number of convolution kernels of each size of the at least two sizes according to the preset allocation strategy comprises:
determining the number of convolution kernels of each size of the at least two sizes by taking a scale factor as an allocation basis;
wherein the taking the scale factor as the allocation basis comprises: a proportion of small-size convolution kernels for feature extraction increases along with an increase of the scale factor,
wherein the scale factor is a pixel ratio at which the image changes from a high resolution to a low resolution.

2. The method of claim 1 , wherein the determining the number of convolution kernels of each size according to the preset allocation strategy comprises:
determining the number of convolution kernels of each size according to a strategy of equal-proportion allocation.

3. The method of claim 1 , wherein the preset objective function comprises:
in response to determining an output result of a neural network and a ground truth meet a preset condition based on a norm, the preset objective function performs constraint on sparsity of convolution kernels of a network layer under a corresponding channel through a sparsity function.

4. The method of claim 3 , wherein the preset objective function comprises:

min
W

⁢

(

1
2

⁢



Y
-

W
⁡

(
x
)



1

)

,

s
.
t
.



W

i
,
j



0

≤

k

i
,
j

;

min
W

⁢

(

1
2

⁢



Y
-

W
⁡

(
x
)



2

)

,

s
.
t
.



W

i
,
j



0

≤

k

i
,
j

;

wherein Y represents the ground truth, W(x) represents the output result of the neural network, W i,j represents an element of a j-th convolution kernel of an i-th layer, ∥•∥ 0 represents a zero norm; k i,j represents the sparsity of the j-th convolution kernel of the i-th layer under the corresponding channel; and

min
W

⁢

(
A
)

,

s
·
t

⁢


B


0

represents solving a minimum value of expression A in response to that constraint B is met, wherein i and j are positive integers.

5. The method of claim 3 further comprises: training the preset objective function by at least one of following methods:
a method of separation of variables, or a method of stochastic gradient descent.

6. The method of claim 1 , wherein the performing feature extraction on the image based on the convolution kernels subjected to the sparsity constraint comprises:
deploying the convolution kernels subjected to the sparsity constraint through the preset objective function to an acceleration platform of executing a neural network or an accelerator of executing a neural network; and
performing

Assignees

• Sanechips Tech Co Ltd

Inventors

• Kong Dehui
• Xu Ke
• Zhang Xiao
• Wang Hong
• Han Bin
• Wang Ning
• Liu Xin
• Lu Guoning
• Zhu Fang

Classifications

• G06T3/4046

  using neural networks · CPC title

• G06F18/2136

  based on sparsity criteria, e.g. with an overcomplete basis · CPC title

• G06T3/4053 Primary

  based on super-resolution, i.e. the output image resolution being higher than the sensor resolution · CPC title