Testing rendering of screen objects

Patent metadata

Field	Value
Publication number	US-11647249-B2
Application number	US-201917267212-A
Country	US
Kind code	B2
Filing date	Aug 9, 2019
Priority date	Aug 10, 2018
Publication date	May 9, 2023
Grant date	May 9, 2023

Abstract

The present disclosure relates to methods and devices for testing video data being rendered at or using a media device. A plurality of video frames to be rendered is received, each frame comprising one or more primary screen objects and at least one further screen object. The received frames are rendered at or using the media device wherein the at least one further screen object is superimposed on the one or more primary screen objects of a given frame during rendering. The rendered frames are provided to a data model. Extracted metadata indicating the presence or absence of further screen objects in the rendered video frames is the output of the data model. The data model is also provided with original metadata associated with the video frames prior to rendering. The rendering of each further screen object is then tested based on the original metadata and extracted metadata relating to a given video frame. The disclosure also extends to associated methods and devices for generating training data for testing rendering of video frame and training a data model using the training data.

First claim

The invention claimed is:

1. A method comprising:
providing, by one or more processors, a media device with a plurality of video frames to be rendered in accordance with first metadata, each frame including one or more primary screen objects and at least one secondary screen object, the media device causing each video frame in the plurality of video frames to be rendered based on the first metadata with superposition of the at least one secondary screen object of that video frame onto the one or more primary screen objects of that video frame;
inputting, by the one or more processors, the rendered plurality of video frames into a data model trained to indicate, for each inputted video frame, whether any secondary screen object is present in that inputted video frame, the data model being trained based on operations that include:
inputting, into the data model, first training data that includes a reference plurality of reference video frames; and
inputting, into the data model, second training data that includes reference data items associated with the reference plurality of reference video frames, each reference data item indicating whether any secondary screen object is present in a corresponding reference video frame among the reference plurality of reference video frames;

obtaining, by the one or more processors and from the data model, second metadata that indicates, for each inputted video frame, whether any secondary screen object is present in that inputted video frame;
causing, by the one or more processors, a comparison of the second metadata obtained from the data model to the first metadata in accordance with which the plurality of video frames was rendered; and
providing, by the one or more processors and based on the comparison of the second metadata to the first metadata, a validation result that indicates whether the at least one secondary screen objects in the plurality of video frames were rendered correctly.

2. The method of claim 1 , wherein:
the data model is trained based on the operations, further including:
obtaining, from the data model, extracted metadata that indicates, for each reference video frame, whether any secondary screen object is present in that reference video frame.

3. The method of claim 2 , wherein:
the data model is trained based on the operations, further including:
applying at least one function that reduces error between the extracted metadata from the data model and the reference data items associated with the reference plurality of reference video frames.

4. The method of claim 1 , wherein:
the data model is trained based on the operations, further including:
causing a reference device to produce the reference plurality of reference video frames by producing a first rendering of a reference video stream with secondary screen objects visible;
causing the reference device to produce a comparison plurality of reference video frames by producing a second rendering of the reference video stream without secondary screen objects visible; and
obtaining the reference data items of the second training data by comparing the first rendering of the reference video stream with secondary screen objects visible to the second rendering of the reference video stream without secondary screen objects visible.

5. The method of claim 1 , further comprising:
generating the validation result based on operations that include:
identifying a characteristic of a secondary screen object in a video frame among the plurality of video frames;
comparing the identified characteristic to a corresponding characteristic represented in the first metadata in accordance with which the plurality of video frames was rendered; and
detecting a variance in the identified characteristic based on the comparing of the identified characteristic to the corresponding characteristic.

6. The method of claim 5 , further comprising:
responsive to the variance being detected, calculating an offset based on the detected variance; and
causing the media device to adjust a subsequent rendering of the plurality of video frames based on the offset.

7. The method of claim 1 , wherein:
the reference plurality of reference video frames includes pairs of reference video frames, each pair of the pairs including a corresponding first reference video frame rendered with a corresponding secondary screen object for that pair and a corresponding second reference video frame rendered without the corresponding secondary screen object for that pair.

8. A system comprising:
one or more processors; and
a memory storing instructions that, when executed by at least one processor among the one or more processors, cause the system to perform system operations comprising:
providing a media device with a plurality of video frames to be rendered in accordance with first metadata, each frame including one or more primary screen objects and at least one secondary screen object, the media device causing each video frame in the plurality of video frames to be rendered based on the first metadata with superposition of the at least one secondary screen object of that video frame onto the one or more primary screen objects of that video frame;
inputting the rendered plurality of video frames into a data model trained to indicate, for each inputted video frame, whether any secondary screen object is present in that inputted video frame, the data model being trained based on operations that include:
inputting, into the data model, first training data that includes a reference plurality of reference video frames; and
inputting, into the data model, second training data that includes reference data items associated with the reference plurality of reference video frames, each reference data item indicating whether any secondary screen object is present in a corresponding reference video frame among the reference plurality of reference video frames;

obtaining, from the data model, second metadata that indicates, for each inputted video frame, whether any secondary screen object is present in that inputted video frame;
causing a comparison of the second metadata obtained from the data model to the first metadata in accordance with which the plurality of video frames was rendered; and
providing, based on the comparison of the second metadata to the first metadata, a validation result that indicates whether the at least one secondary screen objects in the plurality of video frames were rendered correctly.

9. The system of claim 8 , wherein:
the data model is trained based on the training operations, further including:
obtaining, from the data model, extracted metadata that indicates, for each reference video frame, whether any secondary screen object is present in that reference video frame.

10. The system of claim 9 , wherein:
the data model is trained based on the training operations, further including:
applying at least one function that reduces error between the extracted metadata from the data model and the reference data items associated with the reference plurality of reference video frames.

11. The system of claim 8 , wherein:
the data model is trained based on the training operations, further including:
causing a reference device to produce the reference plurality of reference video frames by producing a first rendering of a reference video stream with secondary screen objects visible;
causing the reference device to produce a comparison plurality of reference video frames by producing a second rendering of the reference video stream without secondary screen objects visible; and
obtaining the reference data items of the second training data by comparing the first rendering of the r

Assignees

• Nagravision Sa

Inventors

• Gore Douglas
• Zou Ping

Classifications

• H04N21/2187

  Live feed · CPC title

• H04N21/84

  Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} · CPC title

• H04N17/04

  for receivers · CPC title

• H04N21/44008

  involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• H04N21/4425

  Monitoring of client processing errors or hardware failure · CPC title