Electronic apparatus and image processing method thereof

Patent metadata

Field	Value
Publication number	US-11861809-B2
Application number	US-202117565938-A
Country	US
Kind code	B2
Filing date	Dec 30, 2021
Priority date	May 2, 2019
Publication date	Jan 2, 2024
Grant date	Jan 2, 2024

Abstract

An electronic apparatus is disclosed. The electronic apparatus includes a memory storing at least one instruction, and a processor, electrically connected to the memory, configured to, by executing the instruction, obtain, from an input image, a noise map corresponding to the input image; provide the input image to an input layer of a learning network model including a plurality of layers, the learning network model being an artificial intelligence (AI) model that is obtained by learning, through an AI algorithm, a relationship between a plurality of sample images, a respective noise map of each of the plurality of sample images, and an original image corresponding to the plurality of sample images; provide the noise map to at least one intermediate layer among the plurality of layers; and obtain an output image based on a result from providing the input image and the noise map to the learning network model.

First claim

What is claimed is:

1. An electronic apparatus comprising:
a memory storing one or more instructions; and
a processor, electrically connected to the memory, and configured to execute the one or more instructions to:
obtain an input image;
provide the input image to an artificial intelligence (AI) model including a plurality of layers, the AI model being obtained by learning, through an AI algorithm, a first relationship between a plurality of sample noise images and an original image corresponding to the plurality of sample noise images, and a second relationship between a plurality of sample upscaled images and an original image corresponding to the plurality of sample upscaled images; and
obtain an output image in which the input image is processed in a frame by frame manner by the AI model,
wherein the output image is an upscaled version of the input image, and has less noise than the input image.

2. The electronic apparatus of claim 1 ,
wherein the processor is further configured to obtain noise information by providing the input image to a noise information generation model including a plurality of layers, and
wherein the noise information generation model is obtained by learning a relationship between a plurality of sample noise images and a respective noise information of each of the plurality of sample noise images.

3. The electronic apparatus of claim 2 ,
wherein the processor is further configured to provide the noise information to each of the plurality of layers or provide the noise information to each of remaining layers except an input layer among the plurality of layers.

4. The electronic apparatus of claim 1 ,
wherein the AI model comprises a first AI model obtained by learning, through a first AI algorithm, a relationship between an output image that is obtained by sequentially processing, by the plurality of layers, each of a plurality of sample noise images provided to an input layer, among the plurality of layers, a respective noise information of each of the plurality of sample noise images provided to the plurality of intermediate layers and an original image corresponding to each of the plurality of sample noise images.

5. The electronic apparatus of claim 1 ,
wherein the AI model comprises a first AI model obtained by learning a relationship between an original image and a sample image obtained by adding noise to the original image.

6. The electronic apparatus of claim 1 ,
wherein the AI model comprises a second AI model obtained by learning a relationship between an original image and an upscaled sample image obtained after lowering a resolution of the original image.

7. The electronic apparatus of claim 1 , further comprising: obtain, from the input image, a noise information corresponding to the input image; provide the noise information directly to one or more of a plurality of intermediate layers, among the plurality of layers.

8. The electronic apparatus of claim 7 , further comprising: provide the noise information directly to each of the plurality of intermediate layers, among the plurality of layers.

9. An image processing method of an electronic apparatus, the method comprising:
obtaining an input image;
providing the input image to an artificial intelligence (AI) model including a plurality of layers, the AI model being obtained by learning, through an AI algorithm, a first relationship between a plurality of sample noise images and an original image corresponding to the plurality of sample noise images, and a second relationship between a plurality of sample upscaled images and an original image corresponding to the plurality of sample upscaled images; and
obtaining an output image in which the input image is processed in a frame by frame manner by the AI model,
wherein the output image is an upscaled version of the input image, and has less noise than the input image.

10. The method of claim 9 ,
wherein the obtaining the noise information comprises obtaining the noise information by applying the input image to a noise information generation model including a plurality of layers, and
wherein the noise information generation model is obtained by learning a relationship between a plurality of sample noise images and a respective noise information of each of the plurality of sample noise images.

11. The electronic apparatus of claim 10 ,
wherein the providing the noise information comprises providing the noise information to each of the plurality of layers or provide the noise information to each of remaining layers except an input layer among the plurality of layers.

12. The electronic apparatus of claim 9 ,
wherein the AI model comprises a first AI model obtained by learning, through a first AI algorithm, a relationship between an output image that is obtained by sequentially processing, by the plurality of layers, each of a plurality of sample noise images provided to an input layer, among the plurality of layers, a respective noise information of each of the plurality of sample noise images provided to the plurality of intermediate layers and an original image corresponding to each of the plurality of sample noise images.

13. The method of claim 9 ,
wherein the AI model comprises a first AI model obtained by learning a relationship between an original image and a sample image obtained by adding noise to the original image.

14. The method of claim 9 ,
wherein the AI model comprises a second AI model obtained by learning a relationship between an original image and an upscaled sample image obtained after lowering a resolution of the original image.

15. The electronic apparatus of claim 9 , further comprising: obtain, from the input image, a noise information corresponding to the input image; provide the noise information directly to one or more of a plurality of intermediate layers, among the plurality of layers.

16. The electronic apparatus of claim 15 , further comprising: provide the noise information directly to each of the plurality of intermediate layers, among the plurality of layers.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Park Seungho
• Moon Youngsu

Classifications

• G06T7/194

  involving foreground-background segmentation · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T5/002 Primary

  Physics · mapped topic

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T2207/20081

  Training; Learning · CPC title