Training an encrypted video stream network scoring system with non-reference video scores

Patent metadata

Field	Value
Publication number	US-11405695-B2
Application number	US-202016842676-A
Country	US
Kind code	B2
Filing date	Apr 7, 2020
Priority date	Apr 8, 2019
Publication date	Aug 2, 2022
Grant date	Aug 2, 2022

Abstract

At least three uses of the technology disclosed are immediately recognized. First, a video stream classifier can be trained that has multiple uses. Second, a trained video stream classifier can be applied to monitor a live network. It can be extended by the network provider to customer relations management or to controlling video bandwidth. Third, a trained video stream classifier can be used to infer bit rate switching of codecs used by video sources and content providers. Bit rate switching and resulting video quality scores can be used to balance network loads and to balance quality of experience for users, across video sources. Balancing based on bit rate switching and resulting video quality scores also can be used when resolving network contention.

First claim

We claim as follows:

1. A non-transitory computer readable media impressed with program instructions that, when executed on hardware, cause the hardware to perform steps of building a training data set and training a video stream classifier to assign video quality scores to delivered video streams, the steps including:
selecting a plurality of video examples from a video source;
causing the video source to deliver video streams of the selected video examples, while synthetically impairing packet delivery, including setting available bandwidth,
causing the video source to modify codec transmission parameters by the synthetically impairing packet delivery;
recording network conditions, including measuring at least actual delivered bit rate during the video streams delivery;
directing the video streams of the selected video examples to a receiving device;
recording video frames rendered by the receiving device from the selected video examples;
building a training data set by scoring at least some of the recorded video frames to produce scores and correlating the scores with the recorded network conditions;
using the scores, an identifier of the video source, and the correlated network conditions as a ground truth for training the video stream classifier to assign video scores without dependence on rendering the video streams; and
saving parameters of the trained video stream classifier for use on a live network.

2. The non-transitory computer readable media of claim 1 , further including instructions, that when executed, cause the hardware to perform the steps, wherein:
the video frames rendered by the receiving device are accessed via an HDMI connection.

3. The non-transitory computer readable media of claim 1 , further including instructions, that when executed, cause the hardware to perform the steps, including:
scoring the recorded video frames using a non-reference video classifier that performs the scoring without dependence on access to a reference version, for quality comparison, of the recorded video frames.

4. The non-transitory computer readable media of claim 1 , further including instructions, that when executed, cause the hardware to perform the steps, including:
selecting the video examples to include variety of scene types that vary in image complexity, lighting and color.

5. The non-transitory computer readable media of claim 1 , further including instructions, that when executed, cause the hardware to perform the steps, including:
applying the steps in claim 1 to a plurality of receiving devices of different brands and models and using the receiving device brand and model as elements of the ground truth for the training.

6. A method of building the training data set and training the video stream classifier to assign video quality scores to the delivered video streams, the method including executing program instructions from the non-transitory computer readable media of claim 1 on the hardware.

7. A device configurable to build the training data set and train the video stream classifier to assign video quality scores to the delivered video streams, the device including the non-transitory computer readable media of claim 1 and the hardware adapted to execute the program instructions.

8. A method of building the training data set and training the video stream classifier to assign video quality scores to the delivered video streams, the method including executing program instructions from the non-transitory computer readable media of claim 4 on the hardware.

9. A device configurable to build the training data set and train the video stream classifier to assign video quality scores to the delivered video streams, the device including the non-transitory computer readable media of claim 4 and the hardware adapted to execute the program instructions.

10. A method of building the training data set and training the video stream classifier to assign video quality scores to the delivered video streams, the method including executing program instructions from the non-transitory computer readable media of claim 5 on the hardware.

11. A device configurable to build the training data set and train the video stream classifier to assign video quality scores to the delivered video streams, the device including the non-transitory computer readable media of claim 5 and the hardware adapted to execute the program instructions.

12. The non-transitory computer readable media of claim 1 , further including instructions, that when executed, cause the hardware to perform the steps, including:
measuring network conditions including actual bit rate during delivery of numerous video streams at a plurality of locations on the live network, correlated with data identifying a video source per video stream;
applying the trained video stream classifier to the measured network conditions and the correlated data to assign video quality scores without dependence on rendering images from the video streams;
aggregating the assigned video quality scores based on one or more parameters of the measured network conditions and the identifying data; and
storing at least the aggregated video quality scores.

13. The non-transitory computer readable media of claim 12 , further including instructions, that when executed, cause the hardware to perform the steps, wherein the plurality of locations include 100 to 1,000,000 physical locations on the live network.

14. The non-transitory computer readable media of claim 12 , further including instructions, that when executed, cause the hardware to perform the steps, including:
the numerous video streams at the plurality of locations on the live network further correlated with data identifying a recipient device type per the video stream.

15. The non-transitory computer readable media of claim 12 , further including instructions, that when executed, cause the hardware to perform the steps, including:
the numerous video streams at the plurality of locations on the live network further correlated with data identifying a recipient user per the video stream.

16. The non-transitory computer readable media of claim 12 , further including instructions, that when executed, cause the hardware to perform the steps, including:
raising an alert to a network operating center when the aggregated video quality scores for a portion of the live network reach an alert level.

17. A method of monitoring video quality of delivered video streams on a live network, the method including executing program instructions from the non-transitory computer readable media of claim 12 on the hardware.

18. A device configurable to monitor video quality of delivered video streams on a live network, the device including the program instructions from the non-transitory computer readable media of claim 12 and the hardware adapted to execute the program instructions.

19. A method of monitoring video quality of delivered video streams on a live network, the method including executing program instructions from the non-transitory computer readable media of claim 16 on the hardware.

20. A device configurable to monitor video quality of delivered video streams on a live network, the device including the program instructions from the non-transitory computer readable media of claim 16 and the hardware adapted to execute the program instructions.

Assignees

• Spirent Communications Inc

Inventors

• Colligan Michael
• Bennington Jeremy

Classifications

• H04N21/23418 Primary

  involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• H04N21/64784

  Data processing by the network (data processing in packet switching systems H04L12/56; flow control in packet networks H04L47/10; intermediate storage or scheduling H04L49/90; provisioning of proxy services in data packet switching networks H04L67/56) · CPC title

• H04N21/64738 Primary

  Monitoring network characteristics, e.g. bandwidth, congestion level (data switched network analysis H04L41/14; monitoring functioning in data switched networks H04L43/0817; flow control in packet networks H04L47/10) · CPC title

• H04N21/8547

  involving timestamps for synchronizing content · CPC title