Techniques for predicting perceptual video quality

Patent metadata

Field	Value
Publication number	US-10007977-B2
Application number	US-201514709230-A
Country	US
Kind code	B2
Filing date	May 11, 2015
Priority date	May 11, 2015
Publication date	Jun 26, 2018
Grant date	Jun 26, 2018

Abstract

In one embodiment of the present invention, a quality trainer and quality calculator collaborate to establish a consistent perceptual quality metric via machine learning. In a training phase, the quality trainer leverages machine intelligence techniques to create a perceptual quality model that combines objective metrics to optimally track a subjective metric assigned during viewings of training videos. Subsequently, the quality calculator applies the perceptual quality model to values for the objective metrics for a target video, thereby generating a perceptual quality score for the target video. In this fashion, the perceptual quality model judiciously fuses the objective metrics for the target video based on the visual feedback processed during the training phase. Since the contribution of each objective metric to the perceptual quality score is determined based on empirical data, the perceptual quality score is a more accurate assessment of observed video quality than conventional objective metrics.

First claim

What is claimed is:

1. A computer-implemented method for estimating perceptual video quality, the method comprising:
selecting a set of objective metrics that represent a plurality of deterministic video characteristics;
for each training video included in a set of training videos, receiving a subjective value for a perceptual video quality metric and a set of objective values for the set of objective metrics, wherein the subjective value and the set of objective values describe the training video;
deriving a composite relationship based on a correlation between the subjective value, the set of objective values, and a measure of pixel motion within at least one of the set of training videos, wherein the composite relationship specifies a level of contribution for at least one of the set of objective metrics to the perceptual video quality metric;
for a target video, calculating a first set of values for the set of objective metrics; and
applying the composite relationship to the first set of values to generate an output value for the perceptual video quality metric.

2. The computer-implemented method of claim 1 , wherein deriving the composite relationship comprises performing one or more training operations on the data sets.

3. The computer-implemented method of claim 2 , wherein performing one or more training operations on a given data set comprises applying a support vector machine algorithm or an artificial neural network algorithm to the set of objective values included in the data set.

4. The computer-implemented method of claim 1 , further comprising:
determining that a value included in the first set of values exceeds a predetermined threshold; and
modifying the output value for the perceptual quality metric based on an adjustment factor.

5. The computer-implemented method of claim 1 , further comprising:
computing a motion value based on pixel differences between two consecutive frames of the target video;
determining that the motion value exceeds a predetermined threshold; and
increasing the output value for the perceptual quality metric by a predetermined amount.

6. The computer-implemented method of claim 1 , wherein the set of objective metrics includes at least one of detail loss measure and visual information fidelity.

7. The computer-implemented method of claim 1 , wherein the set of objective metrics includes an anti-noise signal-to-noise ratio, the target video is derived from a source video, and calculating a first value for the anti-noise signal-to-noise ratio comprises:
applying a first low pass filter to the source video;
applying a second low pass filter to the target video that is stronger than the first low pass filter; and
performing one or more signal-to-noise ratio calculations based on the filtered source video and the filtered target video.

8. The computer-implemented method of claim 1 , wherein a first training video included in the set of training videos includes at least one of compressed data and scaled data.

9. The computer-implemented method of claim 1 , wherein a first subjective value for the perceptual video quality metric is a human-observed score for the visual quality of a reconstructed video that is derived from the first training video.

10. A non-transitory computer-readable storage medium including instructions that, when executed by a processing unit, cause the processing unit to estimate perceptual video quality by performing the steps of:
selecting a set of objective metrics that represent a plurality of deterministic video characteristics;
for each training video included in a set of training videos, receiving a subjective value for a perceptual video quality metric and a set of objective values for the set of objective metrics, wherein the subjective value and the set of objective values describe the training video;
deriving a composite relationship based on a correlation between the subjective value, the set of objective values, and a measure of pixel motion within at least one of the set of training videos, wherein the composite relationship specifies a level of contribution for at least one of the set of objective metrics to the perceptual video quality metric;
for a target video, calculating a first set of values for the set of objective metrics; and
applying the composite relationship to the first set of values to generate an output value for the perceptual video quality metric.

11. The non-transitory computer-readable storage medium of claim 10 , wherein deriving the composite relationship comprises performing one or more training operations on the data sets.

12. The non-transitory computer-readable storage medium of claim 10 , further comprising:
computing a motion value based on pixel differences between two consecutive frames of the target video;
determining that the motion value exceeds a predetermined threshold; and
increasing the output value for the perceptual quality metric by a predetermined amount.

13. The non-transitory computer-readable storage medium of claim 10 , wherein a first training video included in the set of training videos includes compressed data derived from a first original video.

14. The non-transitory computer-readable storage medium of claim 13 , wherein a first subjective value for the perceptual video quality metric indicates the variation between a visual quality of the first original video and a visual quality of a reconstructed training video that is derived from the first training video based on one or more decompression operations.

15. The non-transitory computer-implemented method of claim 13 , wherein a first subjective value for the perceptual video quality metric is a human-observed score for the visual quality of a video that is derived from the first training video based on one or more decompression operations.

16. The non-transitory computer-implemented method of claim 1 , wherein the set of objective metrics includes an anti-noise signal-to-noise ratio, the target video is derived from a source video, and calculating a first value for the anti-noise signal-to-noise ratio comprises:
applying a first low pass filter to the source video;
applying a second low pass filter to the target video that is stronger than the first low pass filter; and
performing one or more signal-to-noise ratio calculations based on the filtered source video and the filtered target video.

17. The non-transitory computer-readable storage medium of claim 10 , wherein the composite relationship is an equation.

18. The non-transitory computer-readable storage medium of claim 17 , wherein applying the composite relationship to the first set of values comprises solving the equation for the values included in the first set of values.

19. A system configured to estimate perceptual video quality based on a set of objective metrics that represent a plurality of deterministic video characteristics, the system comprising:
an encoder configured to generate a set of training videos from a plurality of original videos;
a perceptual quality trainer configured to:
for each training video included in a set of training videos, receive a subjective value for a perceptual video quality metric and a set of objective values for the set of objective metrics, wherein the subjective value and the set of objective values describe the training video;
deriving a composite relationship based on a correlation between the subjective value, the set of objective values, and a measure of pixel motion within at least one of the set of training videos, wherein the compos

Assignees

• Netflix Inc

Inventors

• Aaron Anne
• Kim Dae
• Lin Yu-Chieh
• Ronca David
• Schuler Andy
• Tsao Kuyen
• Wu Chi-Hao

Classifications

• H04N21/4666 Primary

  using neural networks, e.g. processing the feedback provided by the user · CPC title

• H04N19/154

  Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147) · CPC title

• G06T9/002 Primary

  using neural networks · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title