Techniques for predicting perceptual video quality

Patent metadata

Field	Value
Publication number	US-10475172-B2
Application number	US-201816017929-A
Country	US
Kind code	B2
Filing date	Jun 25, 2018
Priority date	May 11, 2015
Publication date	Nov 12, 2019
Grant date	Nov 12, 2019

Abstract

In one embodiment of the present invention, a quality trainer and quality calculator collaborate to establish a consistent perceptual quality metric via machine learning. In a training phase, the quality trainer leverages machine intelligence techniques to create a perceptual quality model that combines objective metrics to optimally track a subjective metric assigned during viewings of training videos. Subsequently, the quality calculator applies the perceptual quality model to values for the objective metrics for a target video, thereby generating a perceptual quality score for the target video. In this fashion, the perceptual quality model judiciously fuses the objective metrics for the target video based on the visual feedback processed during the training phase. Since the contribution of each objective metric to the perceptual quality score is determined based on empirical data, the perceptual quality score is a more accurate assessment of observed video quality than conventional objective metrics.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
selecting a set of objective metrics that includes an anti-noise signal-to-noise ratio;
for each training video included in a set of training videos, receiving a subjective value for a perceptual video quality metric and a set of objective values for the set of objective metrics, wherein the subjective value and the set of objective values describe the training video;
deriving a composite relationship based on a correlation between the subjective value, the set of objective values, and a measure of pixel motion within at least one of the set of training videos, wherein the composite relationship specifies a level of contribution for at least one of the set of objective metrics to the perceptual video quality metric;
for a target video, calculating a first set of values for the set of objective metrics; and
applying the composite relationship to the first set of values to generate an output value for the perceptual video quality metric.

2. The computer-implemented method of claim 1 , wherein deriving the composite relationship comprises performing one or more training operations on the data sets.

3. The computer-implemented method of claim 2 , wherein performing one or more training operations on a given data set comprises applying a support vector machine algorithm or an artificial neural network algorithm to the set of objective values included in the data set.

4. The computer-implemented method of claim 1 , further comprising:
determining that a value included in the first set of values exceeds a predetermined threshold; and
modifying the output value for the perceptual quality metric based on an adjustment factor.

5. The computer-implemented method of claim 1 , further comprising:
computing a motion value based on pixel differences between two consecutive frames of the target video;
determining that the motion value exceeds a predetermined threshold; and
increasing the output value for the perceptual quality metric by a predetermined amount.

6. The computer-implemented method of claim 1 , wherein the set of objective metrics further includes at least one of a detail loss measure and a visual information fidelity.

7. The computer-implemented method of claim 1 , wherein the target video is derived from a source video, and calculating a first set of values for the set of objective metrics comprises calculating a first value for the anti-noise signal-to-noise ratio.

8. The computer-implemented method of claim 7 , wherein calculating a first value for the anti-noise signal-to-noise ratio comprises:
applying a first low pass filter to the source video;
applying a second low pass filter to the target video that is stronger than the first low pass filter; and
performing one or more signal-to-noise ratio calculations based on the filtered source video and the filtered target video.

9. The computer-implemented method of claim 1 , wherein a first subjective value for the perceptual video quality metric is a human-observed score for the visual quality of a reconstructed video that is derived from the first training video.

10. The computer-implemented method of claim 1 , wherein a first training video included in the set of training videos includes at least one of compressed data and scaled data.

11. One or more non-transitory computer-readable media including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:
selecting a set of objective metrics that includes an anti-noise signal-to-noise ratio;
for each training video included in a set of training videos, receiving a subjective value for a perceptual video quality metric and a set of objective values for the set of objective metrics, wherein the subjective value and the set of objective values describe the training video;
deriving a composite relationship based on a correlation between the subjective value and the set of objective values, wherein the composite relationship specifies a level of contribution for at least one of the set of objective metrics to the perceptual video quality metric;
for a target video, calculating a first set of values for the set of objective metrics; and
applying the composite relationship to the first set of values to generate an output value for the perceptual video quality metric.

12. The non-transitory computer-readable medium of claim 11 , wherein deriving the composite relationship comprises performing one or more training operations on the data sets.

13. The non-transitory computer-readable medium of claim 12 , wherein performing one or more training operations on a given data set comprises applying a support vector machine algorithm or an artificial neural network algorithm to the set of objective values included in the data set.

14. The non-transitory computer-readable medium of claim 11 , further comprising:
determining that a value included in the first set of values exceeds a predetermined threshold; and
modifying the output value for the perceptual quality metric based on an adjustment factor.

15. The non-transitory computer-readable medium of claim 11 , further comprising:
computing a motion value based on pixel differences between two consecutive frames of the target video;
determining that the motion value exceeds a predetermined threshold; and
increasing the output value for the perceptual quality metric by a predetermined amount.

16. The non-transitory computer-readable medium of claim 11 , wherein the set of objective metrics further includes at least one of a detail loss measure and a visual information fidelity.

17. The non-transitory computer-readable medium of claim 11 , wherein the target video is derived from a source video, and calculating a first set of values for the set of objective metrics comprises calculating a first value for the anti-noise signal-to-noise ratio.

18. The non-transitory computer-readable medium of claim 17 , wherein calculating a first value for the anti-noise signal-to-noise ratio comprises:
applying a first low pass filter to the source video;
applying a second low pass filter to the target video that is stronger than the first low pass filter; and
performing one or more signal-to-noise ratio calculations based on the filtered source video and the filtered target video.

19. The non-transitory computer-readable medium of claim 11 , wherein a first subjective value for the perceptual video quality metric is a human-observed score for the visual quality of a reconstructed video that is derived from the first training video.

20. The non-transitory computer-readable medium of claim 11 , wherein a first training video included in the set of training videos includes at least one of compressed data and scaled data.

21. A system, comprising:
one or more memories including instructions; and
one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of:
selecting a set of one or more objective metrics representing video characteristics;
for each training video included in a set of training videos, receiving a subjective value for a perceptual video quality metric and a set of one or more objective values for the set of one or more objective metrics, wherein the subjective value and the set of objective values describe the training video;
deriving a composite relationship based on a correlation between the subjective value and the set o

Assignees

• Netflix Inc

Inventors

• Aaron Anne
• Kim Dae
• Lin Yu-Chieh
• Ronca David
• Schuler Andy
• Tsao Kuyen
• Wu Chi-Hao

Classifications

• H04N21/466

  Learning process for intelligent management, e.g. learning user preferences for recommending movies {(services using the results of monitoring in broadcast systems H04H60/61)} · CPC title

• G06T7/0002 Primary

  Inspection of images, e.g. flaw detection · CPC title

• H04N21/4666 Primary

  using neural networks, e.g. processing the feedback provided by the user · CPC title

• G06T9/002 Primary

  using neural networks · CPC title

• H04N19/154

  Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147) · CPC title