Error concealment techniques in video decoding

Patent metadata

Field	Value
Publication number	US-9788018-B2
Application number	US-21703908-A
Country	US
Kind code	B2
Filing date	Jun 30, 2008
Priority date	Jun 30, 2008
Publication date	Oct 10, 2017
Grant date	Oct 10, 2017

Abstract

Error concealment techniques for video decoding are described. For example, a video decoder after finding a corrupted picture in a bit stream, finds a suitable neighbor for the corrupted picture. For example, the video decoder favors pictures with the same parity as the corrupted picture and considers picture order count and picture corruption in choosing a neighbor. The decoder then modifies syntax elements for the encoded video in the bit stream to allow the neighbor to be used in concealing the corruption in the corrupted picture. The modification of syntax elements can depend on the particular video decoder implementation. For example, in a software-only multithreaded video decoder, a task graph is modified, while in a system utilizing video acceleration, syntax elements for reference lists are modified.

First claim

We claim:

1. A method for concealing an error in a bitstream of encoded video using a multithreaded video decoder, the method comprising:
in response to detection of an error in a bitstream of encoded video, identifying a suitable neighbor picture to use in error concealment for a current picture that is a current field, including, from a plurality of pictures represented in a decoded picture buffer (“DPB”):
determining a given picture from among the plurality of pictures represented in the DPB, including searching one or more of the pictures represented in the DPB that have parity the same as parity of the current picture, and then, if no suitable neighbor picture to use in error concealment for the current picture has been found, searching one or more of the pictures represented in the DPB that have parity opposite the parity of the current picture;
determining whether the given picture is decoded or set to be decoded; and
determining whether the given picture is found within a predetermined threshold distance from the current picture;

adding, to a task dependency data structure for the multithreaded video decoder, a dependency between a task of the current picture and a task of the suitable neighbor picture; and
after completion of the task of the suitable neighbor picture, performing error concealment that includes using areas of the suitable neighbor picture in place of corrupted areas of the current picture.

2. A method for concealing an error in a bitstream of encoded video during decoding in a video decoder, the method comprising:
in response to detection of an error in a bitstream of encoded video, determining, from one or more pictures represented in a decoded picture buffer (“DPB”), that a suitable neighbor picture to use in error concealment for a current picture exists in the DPB, including:
determining a given picture from among the one or more pictures represented in the DPB; and
determining, based on one or more factors, that the given picture from the DPB is acceptable for use in error concealment as the suitable neighbor picture;

determining that a corrupted slice from the current picture is an I slice, the corrupted slice including one or more corrupted macroblocks;
in response to the determining that the corrupted slice from the current picture is an I slice, changing the I slice to a P slice;
changing one or more parameters for a reference list to include the suitable neighbor picture; and
modifying parameters of the one or more corrupted macroblocks to facilitate error concealment using the suitable neighbor picture during decoding of the current picture.

3. The method of claim 2 , wherein the current picture is a current field of an interlaced video frame, wherein the current field has a parity, and wherein the determining the given picture comprises:
searching those of the one or more pictures represented in the DPB that have parity the same as the parity of the current field; and
if no suitable neighbor picture to use in error concealment for the current picture has been found, searching those of the one or more pictures represented in the DPB that have parity opposite the parity of the current field.

4. The method of claim 2 , wherein the determining that the given picture from the DPB is acceptable comprises determining if the given picture from the DPB was skipped during decoding.

5. The method of claim 2 , wherein:
the video decoder is a multithreaded video decoder; and
the method further comprises modifying a task dependency data structure such that error concealment for the current picture is dependent on completion of a decoding task for the suitable neighbor picture.

6. The method of claim 2 , wherein:
the video decoder is implemented on a system utilizing video acceleration for motion compensation, inverse frequency transforms and/or loop filtering; and
the method further comprises verifying that the suitable neighbor picture is available for use in decoding that uses the video acceleration.

7. The method of claim 6 , wherein the corrupted slice from the current picture is a first corrupted slice, wherein a second corrupted slice from the current picture is a P slice, and wherein:
the verifying includes determining if the suitable neighbor picture exists in a reference list for the second corrupted slice or if the reference list for the second corrupted slice can be expanded to include the suitable neighbor picture;
the method further comprises:
when the suitable neighbor picture cannot be included in the reference list for the second corrupted slice, using a first picture from the reference list of the second corrupted slice as the suitable neighbor picture; and
changing reference indices of one or more corrupted macroblocks in the second corrupted slice to refer to the suitable neighbor picture.

8. The method of claim 6 , wherein the corrupted slice from the current picture is a first corrupted slice, wherein a second corrupted slice from the current picture is a B slice, and wherein:
the verifying includes determining if the suitable neighbor picture exists in reference lists for the second corrupted slice or if one of the reference lists for the second corrupted slice can be expanded to included the suitable neighbor picture;
the method further comprises:
when the suitable neighbor picture cannot be included in the reference lists for the second corrupted slice, using a first picture from a first of the reference lists of the second corrupted slice as the suitable neighbor picture; and
changing reference indices of one or more corrupted macroblocks in the second corrupted slice to refer to the suitable neighbor picture.

9. The method of claim 2 , wherein:
the video decoder is implemented on a system utilizing a GPU for motion compensation, intra-prediction, and deblocking;
the method further comprises marking as corrupt the one or more corrupted macroblocks such that the GPU performs, using a GPU shader, an additional error concealment task utilizing the suitable neighbor picture.

10. The method of claim 2 , wherein:
the video decoder is implemented on a system utilizing video acceleration for entropy decoding; and
after the modifying, the one or more corrupted macroblocks refer to the suitable neighbor picture and have zero motion.

11. The method of claim 2 , wherein the one or more factors include parity of the current picture and the one or more pictures represented in the DPB.

12. The method of claim 2 wherein, for a given corrupted macroblock of the one or more corrupted macroblocks, the parameters include macroblock type, one or more reference indices and one or more motion vectors.

13. The method of claim 2 , wherein the modifying includes, for a given corrupted macroblock of the one or more corrupted macroblocks:
setting macroblock type for the given corrupted macroblock;
changing one or more reference indices of the given corrupted macroblock to refer to the suitable neighbor picture; and
setting motion of the given corrupted macroblock to predicted motion for the given corrupted macroblock.

14. The method of claim 2 , wherein the one or more factors include one or more of the following: predetermined threshold of temporal distance from the current picture, presence of corruption in the given picture from the DPB, distance between the given picture from the DPB and the current picture, and/or whether the given picture is skipped.

15. A video decoding system comprising one or more processors, memory and one or more storage media having stored thereon software for causing the system to perform operations for concealing an

Assignees

• Wu Yongjun
• Thumpudi Naveen
• Dinu Daniel
• Sanders William R
• Microsoft Technology Licensing Llc

Inventors

• Wu Yongjun
• Thumpudi Naveen
• Dinu Daniel
• Sanders William R

Classifications

• H04N19/895 Primary

  in combination with error concealment · CPC title