Pixel error detection system

Patent metadata

Field	Value
Publication number	US-11080835-B2
Application number	US-201916243650-A
Country	US
Kind code	B2
Filing date	Jan 9, 2019
Priority date	Jan 9, 2019
Publication date	Aug 3, 2021
Grant date	Aug 3, 2021

Abstract

A process receives, with a processor, video content. Further, the process splices, with the processor, the video content into a plurality of video frames. In addition, the process splices, with the processor, at least one of the plurality of video frames into a plurality of image patches. Moreover, the process performs, with a neural network, an image reconstruction of at least one of the plurality of image patches to generate a reconstructed image patch. The process also compares, with the processor, the reconstructed image patch with the at least one of the plurality of image patches. Finally, the process determines, with the processor, a pixel error within the at least one of the plurality of image patches based on a discrepancy between the reconstructed image patch and the at least one of the plurality of image patches.

First claim

We claim:

1. A computer program product comprising a non-transitory computer readable storage device having a computer readable program stored thereon, wherein the computer readable program when executed on a computer causes the computer to:
receive, with a processor, video content;
splice, with the processor, the video content into a plurality of video frames;
splice, with the processor, at least one of the plurality of video frames into a plurality of image patches;
perform, with a neural network, an image reconstruction of at least one of the plurality of image patches to generate a reconstructed image patch;
compare, with the processor, the reconstructed image patch with the at least one of the plurality of image patches;
generate, with the processor, an error score based on a discrepancy between the reconstructed image patch and the at least one of the plurality of image patches;
determine, with the processor, whether the error score is an outlier based on a comparison of the error score with a plurality of error scores corresponding to a distribution of error scores for the plurality of image patches; and
determine, with the processor, whether a pixel error is present within the at least one of the plurality of image patches when the error score is determined to be the outlier.

2. The computer program product of claim 1 , wherein the computer is further caused to compare the error score with an error score corresponding to an image patch of a previous video frame, the image patch of the previous video frame having substantially similar coordinates as the at least one of the plurality of image patches.

3. The computer program product of claim 1 , wherein the computer is further caused to classify the at least one of the plurality of image patches as having a pixel error when a difference between the error score and an error score corresponding to the image patch of a previous video frame exceeds a first tolerance threshold.

4. The computer program product of claim 3 , wherein the computer is further caused to annotate the at least one of the plurality of video frames with a visual marker indicating a location of the pixel error.

5. The computer program product of claim 2 , wherein the computer is further caused to automatically correct the pixel error based upon the classification of the at least one of the plurality of image patches as having the pixel error.

6. The computer program product of claim 1 , wherein the pixel error is a pixel discoloration with respect to the at least one of the plurality of image patches.

7. The computer program product of claim 1 , wherein the computer is further caused to train the neural network based on an image database that stores one or more error-free image patches.

8. The computer program product of claim 1 , wherein the video content is selected from the group consisting of: a movie, a television show, a video game, and an animation.

9. A method comprising:
receiving, with a processor, video content;
splicing, with the processor, the video content into a plurality of video frames;
splicing, with the processor, at least one of the plurality of video frames into a plurality of image patches;
performing, with a neural network, an image reconstruction of at least one of the plurality of image patches to generate a reconstructed image patch;
comparing, with the processor, the reconstructed image patch with the at least one of the plurality of image patches;
generating, with the processor, an error score based on a discrepancy between the reconstructed image patch and the at least one of the plurality of image patches;
determining, with the processor, whether the error score is an outlier based on a comparison of the error score with a plurality of error scores corresponding to a distribution of error scores for the plurality of image patches; and
determining, with the processor, whether a pixel error is present within the at least one of the plurality of image patches when the error score is determined to be the outlier.

10. The method of claim 9 , further comprising comparing the error score with an error score corresponding to an image patch of a previous video frame, the image patch of the previous video frame having substantially similar coordinates as the at least one of the plurality of image patches.

11. The method of claim 9 , further comprising classifying the at least one of the plurality of image patches as having a pixel error when a difference between the error score and an error score corresponding to the image patch of a previous video frame exceeds a first tolerance threshold.

12. The method of claim 11 , further comprising annotating the at least one of the plurality of video frames with a visual marker indicating a location of the pixel error.

13. The method of claim 10 , further comprising automatically correcting the pixel error based upon the classification of the at least one of the plurality of image patches as having the pixel error.

14. The method of claim 9 , wherein the pixel error is a pixel discoloration with respect to the at least one of the plurality of image patches.

15. The method of claim 9 , further comprising training the neural network based on an image database that stores one or more error-free image patches.

16. An apparatus comprising:
a processor that receives video content, splices the video content into a plurality of video frames, splices at least one of the plurality of video frames into a plurality of image patches, compares a reconstructed image patch with the at least one of the plurality of image patches, generates an error score based on a discrepancy between the reconstructed image patch and the at least one of the plurality of image patches, determines whether the error score is an outlier based on a comparison of the error score with a plurality of error scores corresponding to a distribution of error scores for the plurality of image patches, and determines whether a pixel error is present within the at least one of the plurality of image patches when the error score is determined to be the outlier; and
a neural network that performs an image reconstruction of the at least one of the plurality of image patches to generate the reconstructed image patch.

17. The computer program product of claim 3 , wherein the first tolerance threshold is smaller when the at least one of the plurality of image patches is part of the face than when the at least one of the plurality of image patches is not part of the face.

18. The method of claim 11 , wherein the first tolerance threshold is smaller when the at least one of the plurality of image patches is part of the face than when the at least one of the plurality of image patches is not part of the face.

19. The apparatus of claim 16 , wherein the processor further classifies the at least one of the plurality of image patches as having a pixel error when a difference between the error score and an error score corresponding to the image patch of a previous video frame exceeds a first tolerance threshold.

20. The apparatus of claim 19 , wherein the first tolerance threshold is smaller when the at least one of the plurality of image patches is part of the face than when the at least one of the plurality of image patches is not part of the face.

Assignees

• Disney Entpr Inc

Inventors

• Doggett Erika
• Wolak Anna
• Tsatsoulis Penelope Daphne
• Mccarthy Nicholas
• Mandt Stephan

Classifications

• G06V20/40

  in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06T7/0002 Primary

  Inspection of images, e.g. flaw detection · CPC title

• G06V20/41

  Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49) · CPC title