Systems and methods for distortion removal at multiple quality levels

Patent metadata

Field	Value
Publication number	US-10832383-B2
Application number	US-201816167388-A
Country	US
Kind code	B2
Filing date	Oct 22, 2018
Priority date	Apr 30, 2018
Publication date	Nov 10, 2020
Grant date	Nov 10, 2020

Abstract

Systems and methods for distortion removal at multiple quality levels are disclosed. In one embodiment, a method may include receiving training content. The training content may include original content, reconstructed content, and training distortion quality levels corresponding to the reconstructed content. The reconstructed content may be derived from distorted original content. The method may also include training distortion quality levels corresponding to the reconstructed content. The method may further include receiving an initial distortion removal model. The method may include generating a conditioned distortion removal model by training the initial distortion removal model using the training content. The method may further include storing the conditioned distortion removal model.

First claim

The invention claimed is:

1. A computer-implemented method, comprising:
receiving training content, the training content comprising original content and reconstructed content, wherein the reconstructed content is derived from distorted original content;
generating a plurality of training distortion quality levels based on the reconstructed content;
receiving an initial distortion removal model;
generating a conditioned distortion removal model by training the initial distortion removal model using the training content and the plurality of training distortion quality levels; and
storing the conditioned distortion removal model.

2. The computer-implemented method of claim 1 , further comprising:
receiving target content, wherein the target content has one or more target distortion quality levels, wherein the plurality of training distortion quality levels includes at least one different value compared to the one or more target distortion quality levels; and
applying the conditioned distortion removal model to the target content and the one or more target distortion quality levels to generate corrected target content.

3. The computer-implemented method of claim 2 , wherein the initial distortion removal model or the conditioned distortion removal model comprise one or more user-defined output branches based on the one or more target distortion quality levels.

4. The computer-implemented method of claim 1 , wherein the initial distortion removal model comprises two branches, wherein a first set of the training content corresponding to a first branch passes through a first set of convolutional layers, and a second set of the training content corresponding to a second branch passes through the first set of convolutional layers and a second set of convolutional layers.

5. The computer-implemented method of claim 4 , wherein training the initial distortion removal model using the training content comprises:
applying the first set of the training content to the first set of convolutional layers;
applying at least one of the first set of the training content to the second set of convolution layers;
applying the second set of the training content to the first set of convolutional layers and the second set of convolutional layers; and
when the second branch is conditioned, training the first branch and the second branch with equal weighting applied to the first branch and the second branch.

6. The computer-implemented method of claim 1 , wherein the initial distortion removal model or the conditioned distortion removal model comprises a convolutional neural network.

7. The computer-implemented method of claim 1 , wherein the initial distortion removal model or the conditioned distortion removal model comprises an activation function.

8. The computer-implemented method of claim 1 , wherein the conditioned distortion removal model is trained to remove distortions from the target content.

9. The computer-implemented method of claim 1 , wherein the target content comprises one or more videos.

10. A computer-implemented method, comprising:
receiving target content, wherein the target content has a plurality of target distortion quality levels;
receiving a conditioned distortion removal model, the conditioned distortion removal model having been conditioned by training an initial distortion removal model using training content and a plurality of training distortion quality levels, wherein the training content comprises original content and reconstructed content derived from distorted original content, and wherein the plurality of training distortion quality levels are generated based on the reconstructed content; and
applying the conditioned distortion removal model to the target content and the plurality of target distortion quality levels to generate corrected target content.

11. The computer-implemented method of claim 10 , wherein the conditioned distortion removal model comprises one or more user-defined output branches based on the plurality of target distortion quality levels.

12. The computer-implemented method of claim 11 , wherein the target content is corrected based on a given target distortion quality level of a given target content corresponding to one of the one or more user-defined output branches of the conditioned distortion removal model.

13. The computer-implemented method of claim 10 , wherein the conditioned distortion removal model comprises two branches, wherein a first set of the target content corresponding to a first branch passes through a first set of convolutional layers, and a second set of the target content corresponding to a second branch passes through the first set of convolutional layers and a second set of convolutional layers.

14. The computer-implemented method of claim 10 , wherein the initial distortion removal model or the conditioned distortion removal model comprises a convolutional neural network.

15. The computer-implemented method of claim 10 , wherein the target content comprises one or more of an image or a video.

16. The computer-implemented method of claim 15 , wherein the target content comprises one or more of standard content, high definition (HD) content, ultra HD (UHD) content, 4k UHD content, or 8k UHD content.

17. A system, comprising:
a memory storing one or more instructions; and
one or more processors that execute the one or more instructions to perform the steps of:
obtaining target content, wherein the target content has a plurality of target distortion quality levels;
obtaining a conditioned distortion removal model, the conditioned distortion removal model having been conditioned by training an initial distortion removal model using training content and a plurality of training distortion quality levels, wherein the training content comprises original content and reconstructed content derived from distorted original content, and wherein the plurality of training distortion quality levels are generated based on the reconstructed content; and
applying the conditioned distortion removal model to the target content to generate corrected target content.

18. The system of claim 17 , wherein the conditioned distortion removal model comprises one or more user-defined output branches based on the plurality of target distortion quality levels.

19. The system of claim 18 , wherein the target content is corrected based on a given target distortion quality level of a given target content corresponding to one of the one or more user-defined output branches of the conditioned distortion removal model.

20. The system of claim 17 , wherein the initial distortion removal model or the conditioned distortion removal model comprises a convolutional neural network.

Assignees

• Disney Entpr Inc

Inventors

• Schroers Christopher
• Bamert Mauro
• Doggett Erika
• Mcphillen Jared
• Labrozzi Scott
• Weber Romann

Classifications

• G06N3/048

  Activation functions · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title