Training A Non-Reference Video Scoring System With Full Reference Video Scores

Patent metadata

Field	Value
Publication number	US-2019258902-A1
Application number	US-201816216699-A
Country	US
Kind code	A1
Filing date	Dec 11, 2018
Priority date	Feb 16, 2018
Publication date	Aug 22, 2019
Grant date	—

Abstract

The disclosed technology teaches training a NR VMOS score generator by generating synthetically impaired images from FR video using filters tuned to generate impaired versions and applying a FR VMOS generator to pairs of unimpaired FR images from the FR video and the impaired versions of the FR images to create ground truth scores for the impaired versions. The disclosed method also includes training by machine learning model an image evaluation classifier using the ground truth scores and the impaired versions to generate NR VMOS scores, and storing coefficients of the image evaluation classifier for use as the NR VMOS score generator. Also disclosed is generating a NR VMOS score by invoking the trained NR VMOS score generator, with stored coefficients generated by feeding the trained NR VMOS score generator with images captured from scenes in a video to be scored, and evaluating the images to generate NR VMOS scores.

First claim

We claim as follows:

1 . A tangible non-transitory computer readable storage media impressed with computer program instructions that, when executed on a processor, cause the processor to implement a method of training a no-reference video mean opinion score (abbreviated NR VMOS) score generator, the method including:
generating synthetically impaired images from full reference (abbreviated FR) video, using filters tuned to generate impaired versions of unimpaired FR images from the FR video; applying a FR video mean opinion score (abbreviated FR VMOS) generator to pairs of the unimpaired FR images and the impaired versions of the FR images to create ground truth scores for the impaired versions; training by machine learning model an image evaluation classifier using the ground truth scores and the impaired versions to generate NR VMOS scores; and storing coefficients of the image evaluation classifier for use as the NR VMOS score generator.

2 . The tangible non-transitory computer readable storage media of claim 1 , wherein:
the unimpaired FR images from the FR video are selected from a series of scenes; and the filters tuned to generate impaired versions from the FR video approximate effects of constrained video delivery bandwidth.

3 . The tangible non-transitory computer readable storage media of claim 1 , further including generating 50,000 to 10,000,000 synthetically impaired images for use in the applying and the training.

4 . The tangible non-transitory computer readable storage media of claim 1 , further including generating 100,000 to 1,000,000 synthetically impaired images for use in the applying and the training.

5 . The tangible non-transitory computer readable storage media of claim 1 , wherein the machine learning model is a support vector machine (abbreviated SVM) model.

6 . The tangible non-transitory computer readable storage media of claim 1 , wherein the machine learning model is a convolutional neural network (abbreviated CNN) model.

7 . A computer-implemented method for training a no-reference video mean opinion score (abbreviated NR VMOS) score generator, including executing on a processor the program instructions from the non-transitory computer readable storage media of claim 1 , to implement the generating, applying, training and storing.

8 . A computer-implemented method for training a no-reference video mean opinion score (abbreviated NR VMOS) score generator, including executing on a processor the program instructions from the non-transitory computer readable storage media of claim 2 , to implement the generating, applying, training and storing.

9 . A computer-implemented method for training a no-reference video mean opinion score (abbreviated NR VMOS) score generator, including executing on a processor the program instructions from the non-transitory computer readable storage media of claim 5 , to implement the generating, applying, training and storing.

10 . A computer-implemented method for training a no-reference video mean opinion score (abbreviated NR VMOS) score generator, including executing on a processor the program instructions from the non-transitory computer readable storage media of claim 6 , to implement the generating, applying, training and storing.

11 . A system for training a no-reference video mean opinion score (abbreviated NR VMOS) score generator, the system including a processor, memory coupled to the processor, and computer instructions from the non-transitory computer readable storage media of claim 1 loaded into the memory.

12 . The system of claim 11 , wherein:
the unimpaired FR images from the FR video are selected from a series of scenes; and the filters tuned to generate impaired versions from the FR video approximate effects of constrained video delivery bandwidth.

13 . The system of claim 11 , wherein the machine learning model is a support vector machine (abbreviated SVM) model.

14 . The system of claim 11 , wherein the machine learning model is a convolutional neural network (abbreviated CNN) model.

15 . A tangible non-transitory computer readable storage media impressed with computer program instructions that, when executed a processor, cause the processor to implement a method of generating a no-reference video mean opinion score (abbreviated NR VMOS) using a trained NR VMOS score generator, the method including:
invoking the trained NR VMOS score generator that includes stored coefficients generated by training an image evaluation classifier using unimpaired and impaired images from a full reference (abbreviated FR) video; feeding the trained NR VMOS score generator with at least three images captured from different scenes in a video sequence to be scored; evaluating the at least three images to generate NR VMOS scores; and combining the NR VMOS scores from the least three images to generate a sequence NR VMOS score for the video sequence.

16 . The tangible non-transitory computer readable storage media of claim 15 , wherein the at least three images are separated by at least three seconds of video sequence between respective images.

17 . The tangible non-transitory computer readable storage media of claim 15 , wherein the video sequence NR VMOS score for the video sequence satisfies a predetermined correlation with standards-based FR VMOS scores.

18 . A system for generating a no-reference video mean opinion score (abbreviated NR VMOS) using a trained NR VMOS score generator, the system including a processor, memory coupled to the processor, and computer instructions from the non-transitory computer readable storage media of claim 15 loaded into the memory.

19 . A computer-implemented method for generating a no-reference video mean opinion score (abbreviated NR VMOS) using a trained NR VMOS score generator, including executing on a processor the program instructions from the non-transitory computer readable storage media of claim 15 .

Assignees

• Spirent Communications Inc

Inventors

• Colligan Michael
• Bennington Jeremy

Classifications

• G06T7/0002 Primary

  Inspection of images, e.g. flaw detection · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06N3/08

  Learning methods · CPC title