Method and apparatus for performing artificial intelligence encoding and artificial intelligence decoding

Patent metadata

Field	Value
Publication number	US-11816872-B2
Application number	US-202117503420-A
Country	US
Kind code	B2
Filing date	Oct 18, 2021
Priority date	Jun 11, 2020
Publication date	Nov 14, 2023
Grant date	Nov 14, 2023

Abstract

An apparatus for performing artificial intelligence (AI) encoding on an image includes: a memory storing one or more instructions; and a processor configured to execute the one or more instructions stored in the memory to: determine a resolution of an original image; when the resolution of the original image is higher than a predetermined value, obtain a first image by performing AI downscaling on the original image via a downscaling deep neural network (DNN); when the resolution of the original image is lower than or equal to the predetermined value, obtain a first image by performing AI one-to-one preprocessing on the original image via a one-to-one preprocessing DNN for upscaling; generate image data by performing first encoding on the first image; and transmit the image data and AI data including information related to the AI downscaling or information related to the AI one-to-one preprocessing.

First claim

What is claimed is:

1. An apparatus for performing artificial intelligence (AI) decoding on an image, the apparatus comprising:
a memory storing one or more instructions; and
at least one processor configured to execute the one or more instructions stored in the memory to:
obtain AI data and image data, the image data generated as a result of first encoding of a first image, and the AI data related to AI downscaling of an original image to the first image or AI one-to-one preprocessing of the original image to the first image;
obtain a second image corresponding to the first image by performing first decoding on the image data;
obtain, based on the AI data, deep neural network (DNN) setting information for AI upscaling of the second image, from among a plurality of pieces of DNN setting information; and
generate a third image by performing the AI upscaling on the second image via an upscaling DNN operating based on the obtained DNN setting information,
wherein the plurality of pieces of DNN setting information are obtained via:
first joint training of the upscaling DNN and a downscaling DNN used for the AI downscaling of the original image, and
second joint training of a one-to-one preprocessing DNN used for the AI one-to-one preprocessing of the original image and the upscaling DNN, the second joint training being performed using DNN setting information for the AI upscaling obtained as a result of the first joint training.

2. The apparatus of claim 1 , wherein the one-to-one preprocessing DNN is configured to enhance features of the original image while maintaining a same resolution as that of the original image.

3. A method for performing artificial intelligence (AI) decoding on an image, the method comprising:
obtaining AI data and image data, the image data generated as a result of first encoding of a first image, and the AI data related to AI downscaling of an original image to the first image or AI one-to-one preprocessing of the original image to the first image;
obtaining a second image corresponding to the first image by performing first decoding on the image data;
obtaining, based on the AI data, deep neural network (DNN) setting information for AI upscaling of the second image, from among a plurality of pieces of DNN setting information; and
generating a third image by performing the AI upscaling on the second image via an upscaling DNN operating based on the obtained DNN setting information,
wherein the plurality of pieces of DNN setting information are obtained via:
first joint training of the upscaling DNN and a downscaling DNN used for the AI downscaling of the original image, and
second joint training of a one-to-one preprocessing DNN used for the AI one-to-one preprocessing of the original image and the upscaling DNN, the second joint training being performed using DNN setting information for the AI upscaling obtained as a result of the first joint training.

4. The method of claim 3 , wherein the one-to-one preprocessing DNN is configured to enhance features of the original image while maintaining a same resolution as that of the original image.

5. A non-transitory computer-readable recording medium having recorded thereon a program for performing the method of claim 3 .

Assignees

• Samsung Electronics Co Ltd

Inventors

• Lee Chaeeun
• Kim Jaehwan
• Park Youngo

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06T9/002 Primary

  using neural networks · CPC title

• G06N3/08

  Learning methods · CPC title

• G06T3/4046

  using neural networks · CPC title