Automatic annotation of video quality impairment training data for generating machine learning models of video quality prediction

Patent metadata

Field	Value
Publication number	US-2021037284-A1
Application number	US-202016942186-A
Country	US
Kind code	A1
Filing date	Jul 29, 2020
Priority date	Jul 29, 2019
Publication date	Feb 4, 2021
Grant date	—

Abstract

Exemplary embodiments are directed to methods and systems for annotating generating training data for video quality impairment detection. A video stream recorder extracts one or more image frames from a reference video stream as it is played. A video image labeler embeds a unique label into each of the one or more extracted image frames. The video stream recorder records the one or more labeled image frames as a labeled video stream. The video stream player then plays the labeled video stream through an impaired communication channel to generate a degraded video stream. A video image comparator compares one or more corresponding frames of the labeled video stream and the degraded video stream to generate one or more difference frames. An impaired image recorder annotates at least one of the one or more difference frames according to a corrupted region of the at least one difference frame.

First claim

What is claimed is:

1 . A method for annotating training data for video quality impairment detection, comprising:
playing, via a video stream player, a video stream over a first communication channel; extracting, via a video stream recorder, one or more image frames from the video stream being played; embedding, via a video image labeler, a unique label into each of the one or more extracted image frames; recording, via a video stream recorder, the one or more labeled image frames as a labeled video stream; inputting, via a transmission noise generator, random noise into a second communication channel to form an impaired communication channel; playing, via the video stream player, the labeled video stream through the impaired communication channel to generate a degraded video, wherein the impaired communication channel distorts one or more elements of the one or more labeled frames of the labeled video stream with the random noise; comparing, via a video image comparator, one or more corresponding frames of the labeled video stream and the degraded video stream to generate one or more difference frames; and annotating, via an impaired image recorder, at least one of the one or more difference frames according to a corrupted region of the at least one difference frame, wherein the corrupted region corresponds to a non-empty pixel of the at least one difference frame.

2 . The method of claim 1 , further comprising:
generating, via the system controller, a graphical interface for selecting the video stream from storage a memory location.

3 . The method of claim 2 , further comprising:
receiving, at a network interface, the video stream over a communication link of a network according to a selection in the graphical interface.

4 . The method of claim 1 , further comprising:
decoding, via the video image recorder player, the one or more image frames of the video stream prior the extraction.

5 . The method of claim 1 , wherein the unique label is one of bar code, a QR code, and an alphanumeric string.

6 . The method of claim 1 , further comprising:
encoding, via the video image labeler, the one or more labeled frames as a new video stream.

7 . The method of claim 1 , further comprising:
generating, via a system controller, one or more signals for controlling the transmission noise generator to change an impedance of the second communication channel by random amounts.

8 . The method of claim 1 , further comprising:
recording, via the video stream recorder, the degraded video stream.

9 . The method of claim 1 , further comprising:
identifying the corrupted region of the at least one difference image by:
applying a window to the one or more difference frames to identify the non-empty pixels;
generating a list of candidate bounding boxes, where each candidate bounding box has an associated window with non-empty pixels; and
filtering the list of candidate bounding boxes by removing any two bounding boxes which overlap another bounding box having a higher pixel density.

10 . A system for annotating training data for video quality impairment detection, comprising:
one or more processing devices configured to:
play, via a video stream player, a reference video stream over a first communication channel;
extract, via a video stream recorder, one or more image frames from the video stream being played;
embed, via a video image labeler, a unique label into each of the one or more extracted image frames;
record, via a video stream recorder, the one or more labeled image frames as a labeled video stream;
input, via a transmission noise generator, random noise into a second communication channel to form an impaired communication channel;
play, via the video stream player, the labeled video stream through the impaired communication channel to generate a degraded video stream, wherein the impaired communication channel distorts one or more elements of the one or more labeled frames of the labeled video stream with the random noise;
compare, via a video image comparator, one or more corresponding frames of the labeled video stream and the degraded video stream to generate one or more difference frames; and
annotate, via an impaired image recorder, at least one of the one or more difference frames according to a corrupted region of the at least one difference frame, wherein the corrupted region corresponds to a non-empty pixel of the at least one difference frame.

11 . The system of claim 10 , wherein the one or more processors is configured to:
compile, in the video stream recorder, the one or more labeled image frames into the labeled video stream.

12 . The system of claim 10 , wherein the one or more processors is configured to:
encode, via the video image labeler, the one or more labeled frames as a new video stream.

13 . The system of claim 10 , wherein the one or more processors is configured to generate, via a system controller, one or more signals for controlling the transmission noise generator to change an impedance of the second communication channel by random amounts.

14 . The system of claim 10 , wherein to identify the corrupted region of the at least one difference image, the one or more processors is configured to:
apply a window to the one or more difference frames to identify the non-empty pixels; generate a list of candidate bounding boxes, where each candidate bounding box has an associated window with non-empty pixels; and filter the list of candidate bounding boxes by removing any two bounding boxes which overlap another bounding box having a higher pixel density.

15 . A non-transitory computer readable medium storing program code for causing one or more processors to perform operations, the operations comprising:
playing, via a video stream player, a video stream over a first communication channel; extracting, via a video stream recorder, one or more image frames from the video stream being played; embedding, via a video image labeler, a unique label into each of the one or more extracted image frames; recording, via a video stream recorder, the one or more labeled image frames as a labeled video stream; inputting, via a transmission noise generator, random noise into a second communication channel to form an impaired communication channel; playing, via the video stream player, the labeled video stream through the impaired communication channel to generate a degraded video stream, wherein the impaired communication channel distorts one or more elements of the one or more labeled frames of the labeled video stream with the random noise; comparing, via a video image comparator, one or more corresponding frames of the labeled video stream and the degraded video stream to generate one or more difference frames; and annotating, via an impaired image recorder, at least one of the one or more difference frames according to a corrupted region of the at least one difference frame, wherein the corrupted region corresponds to a non-empty pixel of the at least one difference frame.

16 . The non-transitory computer readable medium of claim 15 causing the one or more processors to perform operations further comprising:
identifying the corrupted region of the at least one difference image by:
applying a window to the one or more difference frames to identify the non-empty pixels;
generating a list of candidate bounding boxes, where each candidate bounding box has an associated window with non-empty pixels; and
filtering the list of candidate bounding boxes by removing any two bounding boxes which overlap another

Assignees

• Arris Entpr Llc

Inventors

• Brice Pierre A
• Woodruff Wayne D
• Malysheva Olga
• Begley Robert

Classifications

• H04N17/004 Primary

  for digital television systems · CPC title

• G06V20/48

  Matching video sequences · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• H04N21/44008 Primary

  involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• G06V20/20

  in augmented reality scenes · CPC title