Techniques for predicting perceptual video quality

Patent metadata

Field	Value
Publication number	US-2016335754-A1
Application number	US-201514709230-A
Country	US
Kind code	A1
Filing date	May 11, 2015
Priority date	May 11, 2015
Publication date	Nov 17, 2016
Grant date	—

Abstract

In one embodiment of the present invention, a quality trainer and quality calculator collaborate to establish a consistent perceptual quality metric via machine learning. In a training phase, the quality trainer leverages machine intelligence techniques to create a perceptual quality model that combines objective metrics to optimally track a subjective metric assigned during viewings of training videos. Subsequently, the quality calculator applies the perceptual quality model to values for the objective metrics for a target video, thereby generating a perceptual quality score for the target video. In this fashion, the perceptual quality model judiciously fuses the objective metrics for the target video based on the visual feedback processed during the training phase. Since the contribution of each objective metric to the perceptual quality score is determined based on empirical data, the perceptual quality score is a more accurate assessment of observed video quality than conventional objective metrics.

First claim

What is claimed is:

1 . A computer-implemented method for estimating perceptual video quality, the method comprising:
selecting a set of objective metrics that represent a plurality of deterministic video characteristics; for each training video included in a set of training videos, receiving a data set that describes the training video, wherein the data set includes a subjective value for a perceptual video quality metric and a set of objective values for the set of objective metrics; from the data sets, deriving a composite relationship that determines a value for the perceptual video quality metric based on a set of values for the set of objective metrics; for a target video, calculating a first set of values for the set of objective metrics; and applying the composite relationship to the first set of values to generate an output value for the perceptual video quality metric.

2 . The computer-implemented method of claim 1 , wherein deriving the composite relationship comprises performing one or more training operations on the data sets.

3 . The computer-implemented method of claim 2 , wherein performing one or more training operations on a given data set comprises applying a support vector machine algorithm or an artificial neural network algorithm to the set of objective values included in the data set.

4 . The computer-implemented method of claim 1 , further comprising:
determining that a value included in the first set of values exceeds a predetermined threshold; and modifying the output value for the perceptual quality metric based on an adjustment factor.

5 . The computer-implemented method of claim 1 , further comprising:
computing a motion value based on pixel differences between two consecutive frames of the target video; determining that the motion value exceeds a predetermined threshold; and increasing the output value for the perceptual quality metric by a predetermined amount.

6 . The computer-implemented method of claim 1 , wherein the set of objective metrics includes at least one of detail loss measure and visual information fidelity.

7 . The computer-implemented method of claim 1 , wherein the set of objective metrics includes an anti-noise signal-to-noise ratio, the target video is derived from a source video, and calculating a first value for the anti-noise signal-to-noise ratio comprises:
applying a first low pass filter to the source video; applying a second low pass filter to the target video that is stronger than the first low pass filter; and performing one or more signal-to-noise ratio calculations based on the filtered source video and the filtered target video.

8 . The computer-implemented method of claim 1 , wherein a first training video included in the set of training videos includes at least one of compressed data and scaled data.

9 . The computer-implemented method of claim 1 , wherein a first subjective value for the perceptual video quality metric is a human-observed score for the visual quality of a reconstructed video that is derived from the first training video.

10 . A computer-readable storage medium including instructions that, when executed by a processing unit, cause the processing unit to estimate perceptual video quality by performing the steps of:
selecting a set of objective metrics that represent a plurality of deterministic video characteristics; for each training video included in a set of training videos, receiving a data set that describes the training video, wherein the data set includes a subjective value for a perceptual video quality metric and a set of objective values for the set of objective metrics; from the data sets, deriving a composite relationship that determines a value for the perceptual video quality metric based on a set of values for the set of objective metrics; for a target video, calculating a first set of values for the set of objective metrics; and applying the composite relationship to the first set of values to generate an output value for the perceptual video quality metric.

11 . The computer-readable storage medium of claim 10 , wherein deriving the composite relationship comprises performing one or more training operations on the data sets.

12 . The computer-readable storage medium of claim 10 , further comprising:
computing a motion value based on pixel differences between two consecutive frames of the target video; determining that the motion value exceeds a predetermined threshold; and increasing the output value for the perceptual quality metric by a predetermined amount.

13 . The computer-readable storage medium of claim 10 , wherein a first training video included in the set of training videos includes compressed data derived from a first original video.

14 . The computer-readable storage medium of claim 13 , wherein a first subjective value for the perceptual video quality metric indicates the variation between a visual quality of the first original video and a visual quality of a reconstructed training video that is derived from the first training video based on one or more decompression operations.

15 . The computer-implemented method of claim 13 , wherein a first subjective value for the perceptual video quality metric is a human-observed score for the visual quality of a video that is derived from the first training video based on one or more decompression operations.

16 . The computer-implemented method of claim 1 , wherein the set of objective metrics includes an anti-noise signal-to-noise ratio, the target video is derived from a source video, and calculating a first value for the anti-noise signal-to-noise ratio comprises:
applying a first low pass filter to the source video; applying a second low pass filter to the target video that is stronger than the first low pass filter; and performing one or more signal-to-noise ratio calculations based on the filtered source video and the filtered target video.

17 . The computer-readable storage medium of claim 10 , wherein the composite relationship is an equation.

18 . The computer-readable storage medium of claim 17 , wherein applying the composite relationship to the first set of values comprises solving the equation for the values included in the first set of values.

19 . A system configured to estimate perceptual video quality based on a set of objective metrics that represent a plurality of deterministic video characteristics, the system comprising:
an encoder configured to generate a set of training videos from a plurality of original videos; a perceptual quality trainer configured to:
for each training video included in the set of training videos, receive a data set that describes the training video, wherein the data set includes a subjective value for a perceptual video quality metric and a set of objective values for the set of objective metrics;
from the data sets, derive a composite relationship that determines a value for the perceptual video quality metric based on a set of values for the set of objective metrics; and
a perceptual quality calculator configured to:
for a target video, calculate a first set of values for the set of objective metrics; and
apply the composite relationship to the first set of values to generate an output value for the perceptual video quality metric.

20 . The system of claim 19 , wherein deriving the composite relationship comprises performing one or more training operations on the data sets.

21 . A computer-implemented method for esti

Assignees

• Netflix Inc

Inventors

• Aaron Anne
• Kim Dae
• Lin Yu-Chieh
• Ronca David
• Schuler Andy
• Tsao Kuyen
• Wu Chi-Hao

Classifications

• H04N19/154

  Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147) · CPC title

• G06T2207/30168

  Image quality inspection · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• H04N21/4666 Primary

  using neural networks, e.g. processing the feedback provided by the user · CPC title

• G06T7/0002 Primary

  Inspection of images, e.g. flaw detection · CPC title