Control data for motion-constrained tile set

Patent metadata

Field	Value
Publication number	US-9749627-B2
Application number	US-201314021883-A
Country	US
Kind code	B2
Filing date	Sep 9, 2013
Priority date	Apr 8, 2013
Publication date	Aug 29, 2017
Grant date	Aug 29, 2017

Abstract

Control data for a motion-constrained tile set (“MCTS”) indicates that inter-picture prediction processes within a specified set of tiles are constrained to reference only regions within the same set of tiles in previous pictures in decoding (or encoding) order. For example, a video encoder encodes multiple pictures partitioned into tiles to produce encoded data. The encoder outputs the encoded data along with control data (e.g., in a supplemental enhancement information message) that indicates that inter-picture prediction dependencies across tile set boundaries are constrained for a given tile set of one or more of the tiles. A video decoder or other tool receives the encoded data and MCTS control data, and processes the encoded data. Signaling and use of MCTS control data can facilitate region-of-interest decoding and display, transcoding to limit encoded data to a selected set of tiles, loss robustness, parallelism in encoding and/or decoding, and other video processing.

First claim

We claim:

1. A computer system comprising one or more processing units and memory, wherein the computer system implements an encoder system configured to perform operations comprising:
encoding multiple pictures to produce encoded data, wherein each of the multiple pictures is partitioned into multiple tiles; and
outputting the encoded data along with control data that indicates that inter-picture prediction dependencies across specific boundaries are constrained for a given tile set of one or more tiles of the multiple tiles, wherein the given tile set is parameterized in the control data as one or more tile regions covering the one or more tiles of the multiple tiles, and wherein the control data includes:
a count parameter that indicates a count of tile regions in the given tile set; and
for each of the one or more tile regions in the given tile set, syntax elements that indicate the location of the tile region within the multiple pictures.

2. The computer system of claim 1 wherein the one or more tile regions are one or more tile rectangles, and wherein, for a given tile rectangle of the one or more tile rectangles in the given tile set, the syntax elements that indicate the location of the tile region are syntax elements that identify two corners of the given tile rectangle.

3. The computer system of claim 2 wherein the two corners are a top-left corner of the given tile rectangle and a bottom-right corner of the given tile rectangle.

4. The computer system of claim 1 wherein the control data further includes:
an identifier of the given tile set.

5. The computer system of claim 1 wherein the control data is a supplemental enhancement information (“SEI”) message that indicates that inter-picture prediction dependencies across tile set boundaries are constrained for the given tile set.

6. The computer system of claim 1 wherein the operations further comprise:
deciding whether inter-picture prediction dependencies across specific boundaries are to be constrained for the given tile set; and
if so, constraining motion estimation during the encoding such that inter-picture prediction dependencies across specific boundaries are avoided for the given tile set.

7. The computer system of claim 1 wherein the control data further indicates that inter-picture prediction dependencies across specific boundaries are constrained for each of one or more other tile sets of the multiple tiles, and wherein the encoding uses parallel processing for at least some stages of encoding for the given tile set and the one or more other tile sets.

8. The computer system of claim 1 wherein the control data further indicates that inter-picture prediction dependencies across specific boundaries are constrained for each of one or more other tile sets of the multiple tiles, and wherein the encoding configures the given tile set and one or more other tile sets to provide gradual decoder refresh functionality.

9. In a computer system, a method comprising:
receiving encoded data for multiple pictures, wherein each of the multiple pictures is partitioned into multiple tiles;
receiving control data that indicates that inter-picture prediction dependencies across specific boundaries are constrained for a given tile set of one or more tiles of the multiple tiles, wherein the given tile set is parameterized in the control data as one or more tile regions covering the one or more tiles of the multiple tiles, and wherein the control data includes:
a count parameter that indicates a count of tile regions in the given tile set; and
for each of the one or more tile regions in the given tile set, syntax elements that indicate the location of the tile region within the multiple pictures; and

processing the encoded data.

10. The method of claim 9 wherein the one or more tile regions are one or more tile rectangles, and wherein, for a given tile rectangle of the one or more tile rectangles in the given tile set, the syntax elements that indicate the location of the tile region are syntax elements that identify two corners of the given tile rectangle.

11. The method of claim 10 wherein the two corners are a top-left corner of the given tile rectangle and a bottom-right corner of the given tile rectangle.

12. The method of claim 9 wherein the control data further includes:
an identifier of the given tile set.

13. The method of claim 9 wherein the control data is a supplemental enhancement information (“SEI”) message that indicates that inter-picture prediction dependencies across tile set boundaries are constrained for the given tile set.

14. The method of claim 9 wherein the control data further indicates that inter-picture prediction dependencies across specific boundaries are constrained for each of one or more other tile sets of the multiple tiles, and wherein the processing includes decoding that uses parallel processing for at least some stages of decoding for the given tile set and the one or more other tile sets.

15. The method of claim 9 wherein the processing includes:
upon detection of loss of at least some of the encoded data other than the given tile set, decoding of the given tile set as part of loss recovery.

16. One or more computer-readable media storing computer-executable instructions for causing a computer system programmed thereby to perform operations, wherein the one or more computer-readable media are selected from the group consisting of volatile memory, non-volatile memory, magnetic disk, a CD-ROM, and a DVD, the operations comprising:
receiving encoded data for multiple pictures, wherein each of the multiple pictures is partitioned into multiple tiles;
receiving control data that indicates that inter-picture prediction dependencies across specific boundaries are constrained for a given tile set of one or more tiles of the multiple tiles, wherein:
an identifier of the given tile set;
a count parameter that indicates a count of tile regions in the given tile set; and
for each of the tile regions in the given tile set, syntax elements that indicate the location of the tile region within the multiple pictures; and

processing the encoded data.

17. The one or more computer-readable media of claim 16 wherein the control data is a supplemental enhancement information (“SEI”) message that indicates that inter-picture prediction dependencies across tile set boundaries are constrained for the given tile set, and wherein the SEI message includes a syntax element that is usable by a decoder system to assess whether quality may be adversely affected if only the given tile set is decoded.

18. The one or more computer-readable media of claim 16 wherein the control data further indicates that inter-picture prediction dependencies across specific boundaries are constrained for each of one or more other tile sets of the multiple tiles, and wherein the control data further includes:
a parameter that indicates a count of tile sets; and
for each of the one or more other tile sets:
an identifier of the other tile set;
a count parameter that indicates a count of tile regions in the other tile set; and
for each of the tile regions in the other tile set, syntax elements that indicate the location of the tile region within the multiple pictures.

19. The one or more computer-readable media of claim 16 wherein the control data further indicates that inter-picture prediction dependencies across specific boundaries are constrained for each of one or more other tile sets of the multiple tiles, and wherein the processing the encoded data inclu

Assignees

• Microsoft Corp
• Microsoft Technology Licensing Llc

Inventors

• Wu Yongjun
• Sullivan Gary J
• Zhang Yifu

Classifications

• H04N19/577

  Motion compensation with bidirectional frame interpolation, i.e. using B-pictures · CPC title

• G06T7/10

  Segmentation; Edge detection (motion-based segmentation G06T7/215) · CPC title

• H04N19/65

  using error resilience · CPC title

• H04N19/174

  the region being a slice, e.g. a line of blocks or a group of blocks · CPC title

• H04N19/107

  between spatial and temporal predictive coding, e.g. picture refresh · CPC title