Method, electronic device, and computer program product for image processing

Patent metadata

Field	Value
Publication number	US-12141942-B2
Application number	US-202117400382-A
Country	US
Kind code	B2
Filing date	Aug 12, 2021
Priority date	Jul 23, 2021
Publication date	Nov 12, 2024
Grant date	Nov 12, 2024

Abstract

Embodiments of the present disclosure provide a method, an electronic device, and a program product for image processing. In one embodiment, a method may include: at an edge node of a network, obtaining a first image generated based on data associated with a target event, wherein the first image has a first resolution ratio. Additionally, the method may further include: sending a second image converted from the first image to a terminal device, wherein the second image has a second resolution ratio higher than the first resolution ratio. According to the embodiments of the present disclosure, by rendering a low-resolution-ratio image at a cloud server and transmitting the image to an edge node or a terminal device for reconstructing a high-resolution-ratio image, the bandwidth and time delay of high-definition image transmission can be significantly reduced, so that the user experience is improved.

First claim

What is claimed is:

1. A method for image processing, comprising:
at an edge node of a network, obtaining a first image generated based on data associated with a target event, wherein the first image has a first resolution ratio; and
sending a second image converted from the first image to a terminal device, wherein the second image has a second resolution ratio higher than the first resolution ratio;
wherein the second image is converted from the first image utilizing a trained image processing model that is trained in a cloud server of the network, the cloud server being separate from the edge node, the trained image processing model being provided from the cloud server to the edge node for installation in the edge node;
wherein the trained image processing model is trained in the cloud server utilizing a training dataset that comprises images from each of (i) at least one video obtained by the cloud server for a specific type of video game from at least one video content platform external to the cloud server, and (ii) at least one video obtained by the cloud server from execution of the specific type of video game in a gaming engine of the cloud server, the training dataset further comprising one or more images obtained by the cloud server from the terminal device from execution of the specific type of video game at the terminal device;
wherein the first image is provided from the cloud server to the edge node as a rendered first image generated by the cloud server at the first resolution ratio utilizing the gaming engine of the cloud server; and
wherein the converted second image is provided from the edge node to the terminal device at the second resolution ratio higher than the first resolution ratio for presentation by the terminal device at the second resolution ratio.

2. The method according to claim 1 , further comprising:
converting the first image into the second image based on the trained image processing model.

3. The method according to claim 2 , further comprising:
receiving the image processing model from the cloud server of the network, wherein the image processing model is obtained through training by using low-resolution-ratio reference images as inputs and corresponding high-resolution-ratio reference images as outputs.

4. The method according to claim 3 , wherein the low-resolution-ratio reference images and the corresponding high-resolution-ratio reference images are obtained from at least one of the following:
a frame in a video obtained by the cloud server;
an image rendered at the cloud server; and
an image received from the terminal device.

5. The method according to claim 1 , wherein the edge node obtains a third image generated based on additional data associated with the target event, the third image having a third resolution ratio, and sends the third image to the terminal device, and wherein the terminal device converts the received third image into a fourth image, the fourth image having a fourth resolution ratio higher than the third resolution ratio.

6. The method according to claim 1 , wherein the target event comprises at least one of the following: an entertainment interaction event, an online education event, and an online conference event.

7. A computer program product tangibly stored on a non-transitory computer-readable medium and comprising machine-executable instructions, wherein the machine-executable instructions, when executed, cause a machine to perform the method according to claim 1 .

8. The method according to claim 1 , wherein the trained image processing model is fully trained in the cloud server, prior to being provided from the cloud server to the edge node for installation therein, and is not further trained in the edge node.

9. A method for image processing, comprising:
at a terminal device, receiving a first image which is from an edge node of a network and generated based on data associated with a target event, wherein the first image has a first resolution ratio;
converting the first image at the terminal device into a second image, the second image having a second resolution ratio higher than the first resolution ratio; and
presenting the second image at the terminal device;
wherein the second image is converted from the first image utilizing a trained image processing model that is trained in a cloud server of the network, the cloud server being separate from the edge node, the trained image processing model being provided from the cloud server to the terminal device for installation in the terminal device;
wherein the trained image processing model is trained in the cloud server utilizing a training dataset that comprises images from each of (i) at least one video obtained by the cloud server for a specific type of video game from at least one video content platform external to the cloud server, and (ii) at least one video obtained by the cloud server from execution of the specific type of video game in a gaming engine of the cloud server, the training dataset further comprising one or more images obtained by the cloud server from the terminal device from execution of the specific type of video game at the terminal device;
wherein the first image is provided from the cloud server to the terminal device via the edge node as a rendered first image generated by the cloud server at the first resolution ratio utilizing the gaming engine of the cloud server; and
wherein the converted second image is generated in the terminal device at the second resolution ratio higher than the first resolution ratio, for presentation by the terminal device at the second resolution ratio.

10. The method according to claim 9 , wherein converting the first image into the second image comprises:
inputting the first image into the trained image processing model so as to obtain the second image.

11. The method according to claim 10 , further comprising:
receiving the image processing model from the cloud server of the network, wherein the image processing model is obtained through training by using low-resolution-ratio reference images as inputs and corresponding high-resolution-ratio reference images as outputs.

12. The method according to claim 11 , wherein the low-resolution-ratio reference images and the corresponding high-resolution-ratio reference images are obtained from at least one of the following:
a frame in a video obtained by the cloud server;
an image rendered at the cloud server; and
an image received from the terminal device.

13. The method according to claim 9 , wherein the target event comprises at least one of the following: an entertainment interaction event, an online education event, and an online conference event.

14. A computer program product tangibly stored on a non-transitory computer-readable medium and comprising machine-executable instructions, wherein the machine-executable instructions, when executed, cause a machine to perform the method according to claim 7 .

15. An electronic device, comprising:
at least one processor; and
a memory coupled to the at least one processor and having instructions stored therein, wherein the instructions, when executed by the at least one processor, cause the electronic device to perform actions comprising:
at an edge node of a network, obtaining a first image generated based on data associated with a target event, wherein the first image has a first resolution ratio; and
sending a second image converted from the first image to a terminal device, wherein the second image has a second resolution ratio higher than the first resolution ratio;
wherein the second image is converted from the firs

Assignees

• Emc Ip Holding Co Llc

Inventors

• Orellana Pedro Fernandez
• Chen Qiang

Classifications

• G06T3/4092

  Image resolution transcoding, e.g. by using client-server architectures · CPC title

• A63F13/525

  Changing parameters of virtual cameras · CPC title

• G06N20/00

  Machine learning · CPC title

• G06T3/4046

  using neural networks · CPC title

• A63F13/67

  adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use · CPC title