Techniques for detecting performance issues in video processing

Patent metadata

Field	Value
Publication number	US-2021124667-A1
Application number	US-201916667575-A
Country	US
Kind code	A1
Filing date	Oct 29, 2019
Priority date	Oct 29, 2019
Publication date	Apr 29, 2021
Grant date	—

Abstract

Examples described herein generally relate to detecting potential issues in video processing. For each of multiple instances of time over a time period, a set of values corresponding to different performance metrics related to video processing can be obtained. A data structure can be generated to include the set of values. The data structure can be compared to a set of data structures in a convolutional neural network (CNN) model. Based on comparing the data structure to the set of data structures, it can be determined whether the set of values represent a potential issue in video processing or not.

First claim

1 . A computer-implemented method for detecting performance issues in video processing, comprising:
obtaining, for each of multiple instances of time over a time period, a set of values corresponding to different performance metrics related to video processing; generating a data structure as a one-dimensional image including the set of values corresponding to the different performance metrics over the time period stored as pixel values in the one-dimensional image, wherein a first pixel value of a pixel of the one-dimensional image is a first value of the set of values that corresponds to a first performance metric, and wherein a second pixel value of the pixel of the one-dimensional image is a second value of the set of values that corresponds to a second performance metric; comparing the one-dimensional image to a set of one-dimensional images in a convolutional neural network (CNN) model, wherein the set of one-dimensional images have other values for the different performance metrics over the time period stored as pixel values; and determining, based on comparing the one-dimensional image to the set of one-dimensional images, whether the set of values represent a potential issue in video processing or not.

2 . (canceled)

3 . The computer-implemented method of claim 1 , wherein the CNN model indicates, for each of the set of one-dimensional images, whether a given one-dimensional image in the set of one-dimensional images represents the potential issue in video processing or not.

4 . The computer-implemented method of claim 3 , wherein the determining whether the set of values represent the potential issue is based at least in part on determining whether the one-dimensional image is similar to a subset of the set of one-dimensional images indicated as having the potential issue in video processing.

5 . The computer-implemented method of claim 1 , wherein the different performance metrics include a measurement of frames-per-second.

6 . The computer-implemented method of claim 5 , wherein the different performance metrics further include a measurement of central processing unit (CPU) utilization or graphics processing unit (GPU) utilization.

7 . The computer-implemented method of claim 1 , further comprising training the CNN model with the set of one-dimensional images including at least one synthesized data structure or including at least one actual data structure received during performance of the video processing.

8 . The computer-implemented method of claim 7 , further comprising receiving the at least one actual data structure based on feedback from one or more applications.

9 . The computer-implemented method of claim 1 , further comprising reporting, to an interface, the potential issue in video processing.

10 . The computer-implemented method of claim 1 , wherein obtaining the set of values comprises obtaining the set of values from a computing device on which the video processing is occurring, which is different from a device on which the one-dimensional image is compared to the set of one-dimensional images.

11 . A computing device for detecting performance issues in video processing, comprising:
a memory storing one or more parameters or instructions for executing an operating system and a plurality of applications; a display interface coupled with a display device for communicating signals to display visual content on the display device; and at least one processor coupled to the memory and the display interface, wherein the at least one processor is configured to:
obtain, for each of multiple instances of time over a time period, a set of values corresponding to different performance metrics related to video processing;
generate a data structure as a one-dimensional image including the set of values corresponding to the different performance metrics over the time period stored as pixel values in the one-dimensional image, wherein a first pixel value of a pixel of the one-dimensional image is a first value of the set of values that corresponds to a first performance metric, and wherein a second pixel value of the pixel of the one-dimensional image is a second value of the set of values that corresponds to a second performance metric;
compare the one-dimensional image to a set of one-dimensional images in a convolutional neural network (CNN) model, wherein the set of one-dimensional images have other values for the different performance metrics over the time period stored as pixel values; and
determine, based on comparing the one-dimensional image to the set of one-dimensional images, whether the set of values represent a potential issue in video processing or not.

12 . (canceled)

13 . The computing device of claim 11 , wherein the CNN model indicates, for each of the set of one-dimensional images, whether a given one-dimensional image in the set of one-dimensional images represents the potential issue in video processing or not.

14 . The computing device of claim 13 , wherein the at least one processor is configured to determine whether the set of values represent the potential issue based at least in part on determining whether the one-dimensional image is similar to a subset of the set of one-dimensional images indicated as having the potential issue in video processing.

15 . The computing device of claim 11 , wherein the different performance metrics include a measurement of frames-per-second.

16 . The computing device of claim 11 , wherein the at least one processor is further configured to train the CNN model with the set of one-dimensional images including at least one synthesized data structure or including at least one actual data structure received during performance of the video processing.

17 . The computing device of claim 11 , wherein the at least one processor is further configured to report, to an interface, the potential issue in video processing.

18 . The computing device of claim 11 , wherein the at least one processor is configured to obtain the set of values from a device on which the video processing is occurring, which is different from the computing device.

19 . A non-transitory computer-readable medium, comprising code executable by one or more processors for detecting performance issues in video processing, the code comprising code for:
obtaining, for each of multiple instances of time over a time period, a set of values corresponding to different performance metrics related to video processing; generating a data structure as a one-dimensional image including the set of values corresponding to the different performance metrics over the time period stored as pixel values in the one-dimensional image, wherein a first pixel value of a pixel of the one-dimensional image is a first value of the set of values that corresponds to a first performance metric, and wherein a second pixel value of the pixel of the one-dimensional image is a second value of the set of values that corresponds to a second performance metric; comparing the one-dimensional image to a set of one-dimensional images in a convolutional neural network (CNN) model, wherein the set of one-dimensional images have other values for the different performance metrics over the time period stored as pixel values; and determining, based on comparing the one-dimensional image to the set of one-dimensional images, whether the set of values represent a potential issue in video processing or not.

20 . (canceled)

21 . The non-transitory computer-readable medium of claim 19 , wherein th

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Hsu Shu-Wei
• Bailey William Jarrad
• Merry Sebastian Carl

Classifications

• G06F11/3419 Primary

  by assessing time · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F11/3409 Primary

  for performance assessment · CPC title