Method, device, and computer program product for video processing

Patent metadata

Field	Value
Publication number	US-12526384-B2
Application number	US-202318112831-A
Country	US
Kind code	B2
Filing date	Feb 22, 2023
Priority date	Jan 11, 2023
Publication date	Jan 13, 2026
Grant date	Jan 13, 2026

Abstract

Embodiments of the present disclosure relate to a method, a device, and a computer program product for video processing. A method in one embodiment comprises receiving a first portion of a video stream from a sending device, wherein frames in the video stream comprise images of a user, and the first portion has a first resolution. The method further comprises acquiring a pre-trained model associated with the user from a set of candidate models, and generating a conversion model by adjusting the pre-trained model using the first portion. The conversion model is used to convert a second portion of the video stream into a reconstructed video stream, the second portion having a resolution lower than the first resolution and the reconstructed video stream having a resolution higher than a resolution. Such an embodiment can provide high-quality video in a privacy-preserving manner, for example, as part of a secure video service.

First claim

What is claimed is:

1 . A method, comprising:
receiving a first portion of a video stream from a sending device, wherein frames in the video stream comprise images of a first user, and the first portion has a first resolution; acquiring, based on features of the first user extracted from the first portion of the video stream, a pre-trained model associated with the first user from a set of candidate models, wherein a candidate model in the set of candidate models is used to convert an input video into an output video having a higher resolution than that of the input video, the pre-trained model being a particular one of the candidate models that is selected based on similarity between (i) the features of the first user as extracted from the first portion of the video stream and (ii) features of additional users as extracted from one or more additional video streams used to train the candidate models; and generating a conversion model specifically adapted to the first user by adjusting the pre-trained model using the first portion, wherein the conversion model is used to convert a second portion of the video stream into a reconstructed video stream, the second portion having a resolution lower than the first resolution and the reconstructed video stream having a resolution higher than the resolution of the second portion.

2 . The method according to claim 1 , wherein a candidate model in the set of candidate models is trained on a group of persons having a set of similar features, and acquiring the pre-trained model associated with the first user comprises:
extracting the set of features of the first user from the first portion; and selecting the pre-trained model from the set of candidate models based on a similarity in the set of features between a corresponding group of persons targeted by a candidate model in the set of candidate models and the first user.

3 . The method according to claim 1 , wherein generating the conversion model comprises:
acquiring a compressed video for the first portion, the compressed video having a resolution lower than the first resolution; and training the pre-trained model by using the first portion and the compressed video as training data to generate the conversion model.

4 . The method according to claim 1 , further comprising:
converting the second portion into the reconstructed video stream using the conversion model; and sending the reconstructed video stream to a receiving device.

5 . The method according to claim 1 , wherein at least one of the resolution of the second portion and the resolution of the reconstructed video stream varies with time.

6 . The method according to claim 1 , further comprising:
determining an edge device in the vicinity of a receiving device for the video stream; and sending the conversion model to the edge device.

7 . The method according to claim 1 , further comprising:
destroying the conversion model in response to ending of transmission of the video stream.

8 . The method according to claim 1 , further comprising:
acquiring a video dataset having a second resolution, wherein frames in the video dataset comprise images of a person; extracting, for the frames in the video dataset, a set of features of the person included in the frames in the video dataset; dividing the video dataset into a plurality of subsets based on a similarity in the set of features between the frames in the video dataset; and generating, for a subset in the plurality of subsets, a candidate model in the set of candidate models that is associated with the subset based on the subset.

9 . The method according to claim 8 , wherein generating the candidate model associated with the subset comprises:
converting frames in the subset to frames having a third resolution, the third resolution being lower than the second resolution; and generating the candidate model associated with the subset using the frames in the subset and the converted frames having the third resolution as training data.

10 . An electronic device, comprising:
at least one processor; and memory coupled to the at least one processor, wherein the memory has instructions stored therein which, when executed by the at least one processor, cause the electronic device to perform actions comprising: receiving a first portion of a video stream from a sending device, wherein frames in the video stream comprise images of a first user, and the first portion has a first resolution; acquiring, based on features of the first user extracted from the first portion of the video stream, a pre-trained model associated with the first user from a set of candidate models, wherein a candidate model in the set of candidate models is used to convert an input video into an output video having a higher resolution than that of the input video, the pre-trained model being a particular one of the candidate models that is selected based on similarity between (i) the features of the first user as extracted from the first portion of the video stream and (ii) features of additional users as extracted from one or more additional video streams used to train the candidate models; and generating a conversion model specifically adapted to the first user by adjusting the pre-trained model using the first portion, wherein the conversion model is used to convert a second portion of the video stream into a reconstructed video stream, the second portion having a resolution lower than the first resolution and the reconstructed video stream having a resolution higher than the resolution of the second portion.

11 . The electronic device according to claim 10 , wherein a candidate model in the set of candidate models is trained on a group of persons having a set of similar features, and acquiring the pre-trained model associated with the first user comprises:
extracting the set of features of the first user from the first portion; and selecting the pre-trained model from the set of candidate models based on a similarity in the set of features between a corresponding group of persons targeted by a candidate model in the set of candidate models and the first user.

12 . The electronic device according to claim 10 , wherein generating the conversion model comprises:
acquiring a compressed video for the first portion, the compressed video having a resolution lower than the first resolution; and training the pre-trained model by using the first portion and the compressed video as training data to generate the conversion model.

13 . The electronic device according to claim 10 , wherein the actions further comprise:
converting the second portion into the reconstructed video stream using the conversion model; and sending the reconstructed video stream to a receiving device.

14 . The electronic device according to claim 10 , wherein at least one of the resolution of the second portion and the resolution of the reconstructed video stream varies with time.

15 . The electronic device according to claim 10 , wherein the actions further comprise:
determining an edge device in the vicinity of a receiving device for the video stream; and sending the conversion model to the edge device.

16 . The electronic device according to claim 10 , wherein the actions further comprise:
destroying the conversion model in response to ending of transmission of the video stream.

17 . The electronic device according to claim 10 , wherein the actions further comprise:
acquiring a video dataset having a second resolution, wherein frames in the video dataset comprise images of a p

Assignees

• Dell Products Lp

Inventors

• Chen Qiang
• Fernandez Orellana Pedro

Classifications

• H03M7/30

  Compression (speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N); Expansion; Suppression of unnecessary data, e.g. redundancy reduction · CPC title

• H04N7/15

  Conference systems · CPC title

• H04N21/440263

  by altering the spatial resolution, e.g. for displaying on a connected PDA · CPC title

• H04N7/147 Primary

  Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q) · CPC title

• H04N21/234363 Primary

  by altering the spatial resolution, e.g. for clients with a lower screen resolution · CPC title