Method, electronic device, and computer program product for video processing

Patent metadata

Field	Value
Publication number	US-11729349-B2
Application number	US-202117400350-A
Country	US
Kind code	B2
Filing date	Aug 12, 2021
Priority date	Jul 23, 2021
Publication date	Aug 15, 2023
Grant date	Aug 15, 2023

Abstract

Illustrative embodiments include a method, an electronic device, and a computer program product for video processing. In the method, a first group of image frames in a first video having a first resolution is converted into a second group of image frames having a second resolution, the first resolution being higher than the second resolution; a second video having the second resolution is generated based on the second group of image frames; conversion parameters for the second video are determined based on the first group of image frames and the second group of image frames, the conversion parameters being used to convert an image frame in the second group of image frames into an image frame having a third resolution, and the third resolution being higher than the second resolution; and the conversion parameters and the second video are sent to a requester of the first video.

First claim

What is claimed is:

1. A method for video processing, comprising:
converting a first group of image frames in a first video having a first resolution into a second group of image frames having a second resolution, the first resolution being higher than the second resolution;
generating a second video having the second resolution based on the second group of image frames;
determining conversion parameters for the second video based on the first group of image frames and the second group of image frames, the conversion parameters being used to convert an image frame in the second group of image frames into an image frame having a third resolution, and the third resolution being higher than the second resolution; and
sending the conversion parameters and the second video to a requester of the first video;
wherein the conversion parameters are determined responsive to the generation of the second video; and
wherein sending the conversion parameters and the second video to the requester of the first video comprises streaming the conversion parameters in combination with the second video to the requester.

2. The method according to claim 1 , wherein determining the conversion parameters comprises:
if a scene switch is detected in the first video, generating scene metadata, the scene metadata indicating a position of the scene switch in the first video; and
determining a first image frame subset of the first group of image frames and a second image frame subset of the second group of image frames based on the scene metadata, image frames in the first image frame subset and the second image frame subset having the same scene; and
determining the conversion parameters based on the image frames in the first image frame subset and the second image frame subset.

3. The method according to claim 2 , further comprising:
sending the scene metadata to the requester of the first video.

4. The method according to claim 1 , wherein determining the conversion parameters based on the first group of image frames and the second group of image frames comprises:
predicting, according to a conversion model, one image frame in the first group of image frames based on multiple image frames in the second group of image frames, the conversion model being used to convert an image having the second resolution into an image having the third resolution;
training the conversion model based on a difference between the predicted image frame and the one image frame in the first group of image frames; and
determining parameters of the trained conversion model as the conversion parameters.

5. A computer program product tangibly stored on a computer-readable medium and comprising machine-executable instructions, the machine-executable instructions, when executed, causing a machine to perform the method of claim 1 .

6. A method for video processing, comprising:
receiving, from a provider of a first video having a first resolution, a second video having a second resolution converted from the first video and conversion parameters for the second video, the first resolution being higher than the second resolution, and the conversion parameters being determined based on a first group of image frames in the first video and a second group of image frames in the second video;
converting, using the conversion parameters, the second group of image frames into a third group of image frames having a third resolution, the third resolution being higher than the second resolution; and
generating a third video having the third resolution based on the third group of image frames to respond to a request for the first video;
wherein the conversion parameters are determined responsive to the generation of the second video; and
wherein the conversion parameters in combination with the second video are streamed from the provider of the first video.

7. The method according to claim 6 , wherein converting the second group of image frames into the third group of image frames comprises:
receiving scene metadata from the provider, the scene metadata indicating a position of a scene switch in the first video;
determining a second image frame subset having the same scene in the second group of image frames based on the scene metadata; and
converting, using the conversion parameters, the second image frame subset into a third image frame subset having the third resolution, as part of the third group of image frames.

8. The method according to claim 6 , wherein converting, using the conversion parameters, the second group of image frames into the third group of image frames comprises:
applying the conversion parameters to a conversion model, the conversion model being used to convert an image having the second resolution into an image having the third resolution; and
generating, according to the conversion model, one image frame in the third group of image frames based on multiple image frames in the second group of image frames.

9. A computer program product tangibly stored on a computer-readable medium and comprising machine-executable instructions, the machine-executable instructions, when executed, causing a machine to perform the method of claim 6 .

10. An electronic device, comprising:
a processor; and
a memory coupled to the processor, the memory having instructions stored therein that, when executed by the processor, cause the electronic device to execute actions comprising:
converting a first group of image frames in a first video having a first resolution into a second group of image frames having a second resolution, the first resolution being higher than the second resolution;
generating a second video having the second resolution based on the second group of image frames;
determining conversion parameters for the second video based on the first group of image frames and the second group of image frames, the conversion parameters being used to convert an image frame in the second group of image frames into an image frame having a third resolution, and the third resolution being higher than the second resolution; and
sending the conversion parameters and the second video to a requester of the first video;
wherein the conversion parameters are determined responsive to the generation of the second video; and
wherein sending the conversion parameters and the second video to the requester of the first video comprises streaming the conversion parameters in combination with the second video to the requester.

11. The electronic device according to claim 10 , wherein determining the conversion parameters comprises:
if a scene switch is detected in the first video, generating scene metadata, the scene metadata indicating a position of the scene switch in the first video; and
determining a first image frame subset of the first group of image frames and a second image frame subset of the second group of image frames based on the scene metadata, image frames in the first image frame subset and the second image frame subset having the same scene; and
determining the conversion parameters based on the image frames in the first image frame subset and the second image frame subset.

12. The electronic device according to claim 11 , wherein the actions further comprise:
sending the scene metadata to the requester of the first video.

13. The electronic device according to claim 10 , wherein determining the conversion parameters based on the first group of image frames and the second group of image frames comprises:
predicting, according to a conversion model, one image frame in the first group of image frames based on multiple image frames in the second group of image frames, the conversio

Assignees

• Emc Ip Holding Co Llc

Inventors

• Chen Qiang
• Fernandez Orellana Pedro

Classifications

• H04N7/0125 Primary

  one of the standards being a high definition standard · CPC title

• H04N21/238

  Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams {(hybrid fiber coaxial [HFC] networks for downstream channel allocation for video distribution H04L12/2801; flow control in packet networks H04L47/10; real-time communication protocols in data switching networks H04L65/00; scheduling or organising the servicing of application requests H04L67/60)} · CPC title

• H04N21/234363 Primary

  by altering the spatial resolution, e.g. for clients with a lower screen resolution · CPC title

• H04N21/23418

  involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• H04N21/23614

  Multiplexing of additional data and video streams (arrangements for simultaneous broadcast of plural pieces of information H04H20/28) · CPC title