Image providing method and apparatus using artificial intelligence, and display method and apparatus using artificial intelligence

Patent metadata

Field	Value
Publication number	US-12229915-B2
Application number	US-202217895416-A
Country	US
Kind code	B2
Filing date	Aug 25, 2022
Priority date	Aug 13, 2021
Publication date	Feb 18, 2025
Grant date	Feb 18, 2025

Abstract

Provided is an electronic apparatus configured to provide an image based on artificial intelligence (AI), the electronic apparatus including a processor configured to execute one or more instructions stored in the electronic apparatus to obtain a first image by AI-downscaling an original image by a downscaling neural network, obtain first image data by encoding the first image, based on a display apparatus not supporting an AI upscaling function, obtain a second image by decoding the first image data, obtain a third image by AI-upscaling the second image by an upscaling neural network, and provide, to the display apparatus, second image data obtained by encoding the third image.

First claim

What is claimed is:

1. An electronic apparatus configured to provide an image based on artificial intelligence (AI), the electronic apparatus comprising:
at least one processor configured to execute one or more instructions stored in the electronic apparatus to cause the electronic apparatus to:
obtain a first image by AI-downscaling an original image by using a downscaling neural network;
obtain first image data by encoding the first image;
obtain a second image by decoding the first image data;
obtain a third image by AI-upscaling the second image by using an upscaling neural network; and
provide second image data obtained by encoding the third image to a first display apparatus based on the first display apparatus not supporting an AI upscaling function; and
provide the first image data to a second display apparatus based on the second display apparatus supporting the AI upscaling function,

wherein, as original frames included in the original image are sequentially AI-downscaled by using the downscaling neural network, first frames included in the first image are obtained, and
wherein electronic apparatus is further configured to sequentially delete the original frames that the AI downscaling is completed.

2. The electronic apparatus of claim 1 , further comprising a camera configured to obtain the original image.

3. The electronic apparatus of claim 1 , wherein the electronic apparatus is further configured to provide the first image data to a second display apparatus based on the second display apparatus supporting an AI upscaling function.

4. The electronic apparatus of claim 1 , wherein the electronic apparatus is further configured to:
obtain the first image by AI-downscaling the original image by using a downscaling neural network to which predetermined first neural network setting information is applied;
select second neural network setting information corresponding to a supported resolution of the first display apparatus from among a plurality of second neural network setting information; and
obtain the third image by AI-upscaling the second image by using an upscaling neural network to which the selected second neural network setting information is applied.

5. The electronic apparatus of claim 4 , wherein the predetermined first neural network setting information and second neural network setting information corresponding to the predetermined first neural network setting information from among the plurality of second neural network setting information are obtained through a joint training of the downscaling neural network and the upscaling neural network, and
wherein second neural network setting information other than the second neural network setting information corresponding to the predetermined first neural network setting information from among the plurality of second neural network setting information is obtained through an individual training of the upscaling neural network after the joint training.

6. The electronic apparatus of claim 1 , wherein the electronic apparatus is further configured to:
obtain the first image by AI-downscaling the original image by using a downscaling neural network to which predetermined first neural network setting information is applied; and
obtain the third image by AI-upscaling the second image n times, where n is a natural number, through an upscaling neural network, based on a scaling factor of the upscaling neural network to which predetermined second neural network setting information is applied and a supported resolution of the first display apparatus.

7. The electronic apparatus of claim 1 , wherein the electronic apparatus is further configured to:
select first neural network setting information for the AI downscaling from among a plurality of first neural network setting information;
obtain the first image by AI-downscaling the original image through a downscaling neural network to which the selected first neural network setting information is applied;
select second neural network setting information corresponding to the selected first neural network setting information from among a plurality of second neural network setting information; and
obtain the third image by AI-upscaling the second image n times, where n is a natural number, through an upscaling neural network, based on a scaling factor of the upscaling neural network to which the selected second neural network setting information is applied and a supported resolution of the first display apparatus.

8. The electronic apparatus of claim 1 , wherein, as second frames included in the second image are sequentially AI-upscaled by using the upscaling neural network, third frames included in the third image are obtained, and
wherein the electronic apparatus is further configured to, based on third frames included in a group of pictures (GOP) being obtained through the AI upscaling, encode the third frames included in the GOP and delete the third frames included in the GOP that the encoding is completed.

9. The electronic apparatus of claim 1 , wherein the electronic apparatus is further configured to:
receive performance information from the first display apparatus; and
determine, from the performance information, whether an AI upscaling function is supported and the supported resolution of the first display apparatus.

10. An image providing method performed by an electronic apparatus using artificial intelligence (AI), the image providing method comprising:
obtaining a first image by AI-downscaling an original image by using a downscaling neural network;
obtaining first image data by encoding the first image;
obtaining a second image by decoding the first image data;
obtaining a third image by AI-upscaling the second image by using an upscaling neural network; and
providing second image data obtained by encoding the third image to a first display apparatus based on the first display apparatus not supporting an AI upscaling function; and
providing the first image data to a second display apparatus based on the second display apparatus supporting the AI upscaling function,
wherein, as original frames included in the original image are sequentially AI-downscaled by using the downscaling neural network, first frames included in the first image are obtained, and
wherein the original frames that the AI downscaling is completed are sequentially deleted.

11. The image providing method of claim 10 , further comprising:
obtaining the original image by a camera.

12. The image providing method of claim 10 , further comprising:
providing the first image data to a second display apparatus based on the second display apparatus supporting an AI upscaling function.

13. The image providing method of claim 10 , further comprising:
obtaining the first image by AI-downscaling the original image by a downscaling neural network to which predetermined first neural network setting information is applied;
selecting second neural network setting information corresponding to a supported resolution of the first display apparatus from among a plurality of second neural network setting information; and
obtaining the third image by AI-upscaling the second image by an upscaling neural network to which the selected second neural network setting information is applied.

14. The image providing method of claim 13 , wherein the predetermined first neural network setting information and second neural network setting information corresponding to the predetermined first neural network setting information from among the plurality of second neural network setting information are obtained through a joint training of th

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Jaehwan
• Park Youngo
• Lee Jongseok
• Lee Chaeeun
• CHOI Kwangpyo

Classifications

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T3/4046 Primary

  using neural networks · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/045

  Combinations of networks · CPC title

• H04N19/85

  using pre-processing or post-processing specially adapted for video compression · CPC title