Methods and apparatuses for performing artificial intelligence encoding and artificial intelligence decoding on image

Patent metadata

Field	Value
Publication number	US-10825203-B2
Application number	US-201916570057-A
Country	US
Kind code	B2
Filing date	Sep 13, 2019
Priority date	Oct 19, 2018
Publication date	Nov 3, 2020
Grant date	Nov 3, 2020

Abstract

Provided is an artificial intelligence (AI) decoding apparatus includes: a memory storing one or more instructions; and a processor configured to execute the one or more instructions stored in the memory, the processor is configured to: obtain AI data related to AI down-scaling an original image to a first image; obtain image data corresponding to an encoding result on the first image; obtain a second image corresponding to the first image by performing a decoding on the image data; obtain deep neural network (DNN) setting information among a plurality of DNN setting information from the AI data; and obtain, by an up-scaling DNN, a third image by performing the AI up-scaling on the second image, the up-scaling DNN being configured with the obtained DNN setting information, wherein the plurality of DNN setting information comprises a parameter used in the up-scaling DNN, the parameter being obtained through joint training of the up-scaling DNN and a down-scaling DNN, and wherein the down-scaling DNN is used to obtain the first image from the original image.

First claim

What is claimed is:

1. An electronic device for displaying an image by using artificial intelligence (AI), the electronic device comprising:
a display; and
one or more processors, when executing one or more instructions stored in the electronic device, configured to:
receive AI data related to AI down-scaling an original image to a first image through a down-scaling neural network (NN) based on first NN setting information, the first NN setting information being selected from a plurality of first NN setting information that is pre-stored in a server, the AI data comprising a bitrate of an image data generated through an encoding on the first image;
receive the image data;
obtain a second image by decoding the image data;
select, based on the AI data, second NN setting information from a plurality of second NN setting information that is pre-stored in the electronic device;
obtain, by an up-scaling NN, a third image by performing AI up-scaling on the obtained second image, the up-scaling NN being set with the selected second NN setting information; and
provide, on the display, the obtained third image,

wherein the plurality of first NN setting information and the plurality of second NN setting information are obtained through joint training of the down-scaling NN and the up-scaling NN.

2. The electronic device of claim 1 , wherein the AI data comprises information related to the first image,
wherein the one or more processors, when executing the one or more instructions, are further configured to select the second NN setting information mapped to the information related to the first image, based on a mapping relationship between a plurality of image-related information and the plurality of second NN setting information, and
wherein the information related to the first image comprises information related to at least one of a resolution, or a codec type.

3. The electronic device of claim 1 , wherein the selected second NN setting information comprises neural network parameters for at least one convolution layer in the up-scaling NN.

4. The electronic device of claim 1 , wherein the AI data comprises an index indicating the second NN setting information for the AI up-scaling.

5. A method of displaying an image by an electronic device by using artificial intelligence (AI), the method comprising:
receiving image data generated through an encoding on a first image;
receiving AI data related to AI down-scaling an original image to the first image through a down-scaling neural network (NN) based on first NN setting information selected from a plurality of first NN setting information that is pre-stored in a server, the AI data comprising a bitrate of the image data;
obtaining a second image by decoding the image data;
selecting, based on the AI data, second NN setting information from a plurality of second NN setting information that is pre-stored in the electronic device;
obtaining, by an up-scaling NN, a third image by performing AI up-scaling on the obtained second image, the up-scaling NN being set with the selected second NN setting information; and
providing, on a display of the electronic device, the obtained third image,
wherein the plurality of first NN setting information and the plurality of second NN setting information are obtained through joint training of the down-scaling NN and the up-scaling NN.

6. The method of claim 5 , wherein the AI data comprises information related to the first image,
wherein the method further comprises selecting the second NN setting information mapped to the information related to the first image, based on a mapping relationship between a plurality of image-related information and the plurality of second NN setting information, and
wherein the information related to the first image comprises information related to at least one of a resolution, or a codec type.

7. The method of claim 5 , wherein the selected second NN setting information comprises neural network parameters for at least one convolution layer in the up-scaling NN.

8. The method of claim 5 , wherein the AI data comprises an index indicating the second NN setting information for the AI up-scaling.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Jaehwan
• Lee Jongseok
• Jeon Sunyoung
• CHOI Kwangpyo
• Choi Minseok
• Dinh Quockhanh
• Park Youngo

Classifications

• H04N19/172

  the region being a picture, frame or field · CPC title

• H04N19/117

  Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635) · CPC title

• H04N19/132 Primary

  Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking · CPC title

• H04N19/85

  using pre-processing or post-processing specially adapted for video compression · CPC title

• G06V10/82

  using neural networks · CPC title