Electronic device for supporting machine learning-based image processing

Patent metadata

Field	Value
Publication number	US-11954833-B2
Application number	US-202217678557-A
Country	US
Kind code	B2
Filing date	Feb 23, 2022
Priority date	Apr 2, 2021
Publication date	Apr 9, 2024
Grant date	Apr 9, 2024

Abstract

An electronic device is provided, which includes a display; an under display camera (UDC); a processor; and a memory that stores instructions, which when executed, causes the processor to receive a first image from the UDC, generate a second image by lowering a resolution of the first image, obtain a third image representing a difference between the second image and an image corresponding to the second image in which artifacts are not present, generate a fourth image by increasing the resolution of the third image, and generate a fifth image by subtracting the fourth image from the first image.

First claim

What is claimed is:

1. An electronic device, comprising:
a camera;
a display located between a subject to be photographed by the camera and the camera;
a processor connected to the camera and the display; and
a memory that stores instructions, which when executed, cause the processor to:
receive a first image from the camera,
generate a second image by lowering a resolution of the first image from NM to (N/n)(M/m),
put the second image into an artificial intelligence (AI) model learned using a first training image having a resolution of (N/n)(M/m) and a second training image having a resolution of (N/n)(M/m) and making a pair with the first training image as an input value,
obtain a third image from a resultant value output from the AI model representing a difference between the second image and an image corresponding to the second image in which artifacts are not present,
generate a fourth image by increasing a resolution of the third image from (N/n)*(M/m) to N*M, and
generate a fifth image by subtracting the fourth image from the first image.

2. The electronic device of claim 1 , wherein the instructions, when executed, further cause the processor to:
transmit the second image to the AI model provided in an external electronic device through a wireless communication circuit of the electronic device, and
obtain the third image from the AI model of the external electronic device through the wireless communication circuit.

3. The electronic device of claim 1 , wherein the instructions, when executed, further cause the processor to receive data including a Bayer pattern or a red-green-blue (RGB) pattern from the camera as the first image.

4. The electronic device of claim 1 , wherein the camera obtains data including a Bayer pattern or a red-green-blue (RGB) pattern by photographing the subject, and processes the data, and
wherein the instructions further cause the processor to receive the processed data from the camera as the first image.

5. The electronic device of claim 1 , wherein the instructions, when executed, further cause the processor to display the fifth image or a sixth image having a resolution that is lower than the resolution of the fifth image on the display.

6. The electronic device of claim 1 , wherein the instructions, when executed, further cause the processor to perform at least one operation of storing the fifth image in the memory and transmitting the fifth image to an external electronic device through a wireless communication circuit of the electronic device.

7. A method performed by an electronic device, the method comprising:
receiving a first image from a camera of the electronic device;
generating a second image by lowering a resolution of the first image from N*M to (N/n)*(M/m);
putting the second image into an artificial intelligence (AI) model learned using a first training image having a resolution of (N/n)*(M/m) and a second training image having a resolution of (N/n)*(M/m) and making a pair with the first training image as an input value;
obtaining a third image from a resultant value output from the AI model, the third image representing a difference between the second image and an image corresponding to the second image in which artifacts are not present;
generating a fourth image by increasing a resolution of the third image from (N/n)*(M/m) to N*M; and
generating a fifth image by subtracting the fourth image from the first image.

8. The method of claim 7 , wherein obtaining the third image comprises:
transmitting the second image to the AI model provided in an external electronic device through a wireless communication circuit of the electronic device; and
obtaining the third image from the AI model of the external electronic device through the wireless communication circuit.

9. The method of claim 7 , wherein receiving the first image comprises receiving data including a Bayer pattern or a red-green-blue (RGB) pattern from the camera as the first image.

10. The method of claim 7 , wherein the camera obtains data including a Bayer pattern or a red-green-blue (RGB) pattern by photographing the subject, and processes the data, and
wherein receiving the first image comprises receiving the processed data from the camera as the first image.

11. The method of claim 7 , further comprising displaying the fifth image or a sixth image having a resolution that is lower than the resolution of the fifth image on a display of the electronic device.

12. The method of claim 7 , further comprising:
performing at least one operation of storing the fifth image in a memory of the electronic device; and
transmitting the fifth image to an external electronic device through a wireless communication circuit of the electronic device.

13. A non-transitory recording medium that stores instructions, which when executed by a processor of an electronic device, cause the processor to:
receive a first image from a camera of the electronic device;
generate a second image by lowering a resolution of the first image from N*M to (N/n)*(M/m);
put the second image into an artificial intelligence (AI) model learned using a first training image having a resolution of (N/n)*(M/m) and a second training image having a resolution of (N/n)*(M/m) and making a pair with the first training image as an input value;
obtain a third image from a resultant value output from the AI model, the third image representing a difference between the second image and an image corresponding to the second image in which artifacts are not present;
generate a fourth image by increasing a resolution of the third image from (N/n)*(M/m) to N*M; and
generate a fifth image by subtracting the fourth image from the first image.

14. The non-transitory recording medium of claim 13 , wherein receiving the first image comprises receiving data including a Bayer pattern or a red-green-blue (RGB) pattern from the camera as the first image.

15. The non-transitory recording medium of claim 13 , wherein the camera obtains data composed of a Bayer pattern or a red-green-blue (RGB) pattern by photographing the subject, and processes the data, and
wherein receiving the first image comprises receiving the processed data from the camera as the first image.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Choi Woojhon
• Do Wonjoon
• Choi Jaesung

Classifications

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title

• H04N23/815

  for controlling the resolution by using a single image · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/20224

  Image subtraction · CPC title