Video corruption detection

Patent metadata

Field	Value
Publication number	US-2022415037-A1
Application number	US-202217840354-A
Country	US
Kind code	A1
Filing date	Jun 14, 2022
Priority date	Jun 24, 2021
Publication date	Dec 29, 2022
Grant date	—

Abstract

Systems, methods, and non-transitory computer-readable media can be configured to train a machine learning model to identify corrupted frames of videos based on training data including video frames exhibiting corruption that is intentionally generated. A frame of a video can be provided to the trained machine learning model. A score indicating a likelihood that the frame of the video exhibits corruption can be determined based on the trained machine learning model.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
training, by a computing system, a machine learning model to identify corrupted frames of videos based on training data including video frames exhibiting corruption that is intentionally generated; providing, by the computing system, a frame of a video to the trained machine learning model; and determining, by the computing system, a score indicating a likelihood that the frame of the video exhibits corruption based on the trained machine learning model.

2 . The computer-implemented method of claim 1 , further comprising:
generating, by the computing system, corruption in a video to create a corrupted version of the video.

3 . The computer-implemented method of claim 2 , wherein the generating corruption in the video comprises:
modifying, by the computing system, a bitstream associated with the video while the video is playing.

4 . The computer-implemented method of claim 2 , further comprising:
recording, by the computing system, frames of the corrupted version of the video; and recording, by the computing system, frames of an uncorrupted version of the video.

5 . The computer-implemented method of claim 4 , wherein the frames of the corrupted version of the video and the frames of the uncorrupted version of the video are recorded based on a predetermined sampling rate.

6 . The computer-implemented method of claim 4 , further comprising:
transforming, by the computing system, the frames of the corrupted version of the video and the frames of the uncorrupted version of the video, wherein the transforming comprises:
cropping, by the computing system, a frame of the corrupted version of the video so that corruption appearing in the frame is preserved.

7 . The computer-implemented method of claim 4 , further comprising:
converting, by the computing system, each frame of the frames of the corrupted version of the video and the uncorrupted version of the video and an associated label into a data representation for training the machine learning model, the data representation including pixel values of the frame.

8 . The computer-implemented method of claim 7 , wherein the data representation is a multidimensional array or a tensor.

9 . The computer-implemented method of claim 1 , wherein the training data includes paired frames including a first frame and a second frame that are identical except for corruption appearing in the first frame.

10 . The computer-implemented method of claim 1 , further comprising:
selecting, by the computing system, frames of the video, including the frame of the video, at a selected sampling rate; and providing, by the computing system, the selected frames of the video to the machine learning model to score the frames.

11 . A system comprising:
at least one processor; and a memory storing instructions that, when executed by the at least one processor, cause the system to perform: training a machine learning model to identify corrupted frames of videos based on training data including video frames exhibiting corruption that is intentionally generated; providing a frame of a video to the trained machine learning model; and determining a score indicating a likelihood that the frame of the video exhibits corruption based on the trained machine learning model.

12 . The system of claim 11 , further comprising:
generating corruption in a video to create a corrupted version of the video.

13 . The system of claim 12 , wherein the generating corruption in the video comprises:
modifying a bitstream associated with the video while the video is playing.

14 . The system of claim 12 , further comprising:
recording frames of the corrupted version of the video; and recording frames of an uncorrupted version of the video.

15 . The system of claim 14 , further comprising:
converting each frame of the frames of the corrupted version of the video and the uncorrupted version of the video and an associated label into a data representation for training the machine learning model, the data representation including pixel values of the frame.

16 . A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform:
training a machine learning model to identify corrupted frames of videos based on training data including video frames exhibiting corruption that is intentionally generated; providing a frame of a video to the trained machine learning model; and determining a score indicating a likelihood that the frame of the video exhibits corruption based on the trained machine learning model.

17 . The non-transitory computer-readable storage medium of claim 16 , further comprising:
generating corruption in a video to create a corrupted version of the video.

18 . The non-transitory computer-readable storage medium of claim 17 , wherein the generating corruption in the video comprises:
modifying a bitstream associated with the video while the video is playing.

19 . The non-transitory computer-readable storage medium of claim 17 , further comprising:
recording frames of the corrupted version of the video; and recording frames of an uncorrupted version of the video.

20 . The non-transitory computer-readable storage medium of claim 19 , further comprising:
converting each frame of the frames of the corrupted version of the video and the uncorrupted version of the video and an associated label into a data representation for training the machine learning model, the data representation including pixel values of the frame.

Assignees

• Meta Platforms Inc

Inventors

• Zhang Wen
• Borba Andrew William
• Ruff Nicholas Jacob

Classifications

• G06T7/0002

  Inspection of images, e.g. flaw detection · CPC title

• G06N20/00

  Machine learning · CPC title

• G06T2207/20132

  Image cropping · CPC title

• G06V10/98 Primary

  Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns · CPC title

• G06T7/174

  involving the use of two or more images · CPC title