Electronic device, image processing method, and computer-readable recording medium

Patent metadata

Field	Value
Publication number	US-11861808-B2
Application number	US-201916971625-A
Country	US
Kind code	B2
Filing date	Feb 20, 2019
Priority date	Feb 20, 2018
Publication date	Jan 2, 2024
Grant date	Jan 2, 2024

Abstract

An electronic device is disclosed. The electronic device of the disclosure comprises: a memory in which a learned artificial intelligence model is stored; and a processor for inputting an input image to the artificial intelligence model and outputting an enlarged image with increased resolution, wherein the learned artificial intelligence model includes an upscaling module for acquiring the pixel values of interpolated pixels around a cell according to a function having a nonlinearly decreasing symmetric form with reference to an original pixel in the enlarged image, the original pixel corresponding to a pixel of the input image.

First claim

The invention claimed is:

1. An electronic device comprising:
a memory storing a learned artificial intelligence model, and at least one programmed instruction; and
a processor configured to execute the at least one programmed instruction to:
input an input image to the artificial intelligence model,
wherein the learned artificial intelligence model includes an upscaling module configured to calculate a pixel value of an interpolated pixel near an original pixel corresponding to a pixel of the input image based on a Gaussian function in a form which is bilaterally symmetrical and nonlinearly decreases with respect to the original pixel,
wherein the upscaling module calculates the pixel value of the interpolated pixel near a plurality of original pixels based on a ratio at which a plurality of original pixel values are reflected in the pixel value of the interpolated pixel, and the ratio is identified according to distances between the plurality of original pixels and the interpolated pixel, on a plurality of Gaussian functions based on the plurality of original pixels, and
output electronically an enlarged image with increased resolution based on the pixel value of the interpolated pixel calculated by the upscaling module.

2. The electronic device as claimed in claim 1 , wherein
the plurality of original pixels correspond to one pixel of the input image in the enlarged image, at least one of a plurality of pixels adjacent to the one pixel, and a pixel corresponding to at least one of a plurality of pixels which are spaced apart to the one pixel but are adjacent to the plurality of pixels.

3. The electronic device as claimed in claim 2 , wherein
a variance of the Gaussian function is calculated based on a linear function for bilinear interpolation of an upscaling factor, wherein the upscaling factor corresponds a magnification of the enlarged image compared to the input image.

4. The electronic device as claimed in claim 3 , wherein
a variance σ d of the Gaussian function is calculated by

σ
d

⁡

(
s
)

=

sqrt
(

-

d
2

2
⁢

ln
⁡

(

-
4

⁢
d

t
⁡

(
s
)

-
1

+
1

)

)

,

 where s is the upscaling factor, d is an x coordinate of a contact point between the linear function and the Gaussian function, and t(s) is a value calculated by adding 1 to a distance between x intercepts of the Gaussian function.

5. The electronic device as claimed in claim 4 , wherein
the Gaussian function is

f
⁡

(

x
;
s

)

=

exp
⁡

(

-

x
2

2
⁢

σ
⁡

(
s
)

2

)

 and σ d (s)−s*0.1≤σ(s)≤σ d (s)+s*0.1.

6. The electronic device as claimed in claim 1 , wherein
the upscaling module further includes a convolution filter configured to calculate a feature of the input image.

7. An image processing method comprising:
receiving an image;
inputting an input image to a learned artificial intelligence model,
wherein the learned artificial intelligence model includes an upscaling module configured to calculate a pixel value of an interpolated pixel near an original pixel corresponding to a pixel of the input image based on a Gaussian function in a form which is bilaterally symmetrical and nonlinearly decreases with respect to the original pixel, and
wherein the upscaling module calculates the pixel value of the interpolated pixel near a plurality of original pixels based on a ratio at which a plurality of original pixel values are reflected in the pixel value of the interpolated pixel, and the ratio is identified according to distances between the plurality of original pixels and the interpolated pixel, on a plurality of Gaussian functions based on the plurality of original pixels; and
outputting electronically, an enlarged image with increased resolution based on the pixel value of the interpolated pixel calculated by the upscaling module.

8. The image processing method as claimed in claim 7 , wherein
the plurality of original pixels correspond to one pixel of the input image in the enlarged image, at least one of a plurality of pixels adjacent to the one pixel, and a pixel corresponding to at least one of a plurality of pixels which are spaced apart to the one pixel but are adjacent to the plurality of pixels.

9. The image processing method as claimed in claim 8 , wherein
a variance of the Gaussian function is calculated based on a linear function for bilinear interpolation of an upscaling factor, wherein the upscaling factor corresponds a magnification of the enlarged image compared to the input image.

10. The image processing method as claimed in claim 9 , wherein
a variance σ d of the Gaussian function is calculated by

Assignees

• Samsung Electronics Co Ltd

Inventors

• Park Jaeyeon
• Ahn Iljun
• Park Yongsup
• Lee Tammy
• Cheon Minsu

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06T3/4053 Primary

  based on super-resolution, i.e. the output image resolution being higher than the sensor resolution · CPC title

• G06F17/18

  for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)} · CPC title