Efficient coding of source video sequences partitioned into tiles

Patent metadata

Field	Value
Publication number	US-11606574-B2
Application number	US-202016882819-A
Country	US
Kind code	B2
Filing date	May 26, 2020
Priority date	May 31, 2019
Publication date	Mar 14, 2023
Grant date	Mar 14, 2023

Abstract

Techniques are disclosed for coding video data in which frames from a video source are partitioned into a plurality of tiles of common size, and the tiles are coded as a virtual video sequence according to motion-compensated prediction, each tile treated as having respective temporal location of the virtual video sequence. The coding scheme permits relative allocation of coding resources to tiles that are likely to have greater significance in a video coding session, which may lead to certain tiles that have low complexity or low motion content to be skipped during coding of the tiles for select source frames. Moreover, coding of the tiles may be ordered to achieve low coding latencies during a coding session.

First claim

We claim:

1. A video coding system, comprising:
a partitioner having an input for a frame from a source video and an output for tiles generated from the frame, each tile having a common size,
a predictive video coder having an input for the tiles and an output for coded tile data coded as a virtual video sequence, wherein the tiles of the frame are associated with different time stamps in the virtual video sequence, and the tiles corresponding to a first tile location over a plurality of frames of the video source are designated as a reference tile at a first frequency lower than tiles corresponding to a second tile location over the plurality of frames, and
a transmitter, having an input for the coded tile data.

2. The system of claim 1 , wherein the coded tile data contains data representing a spatial location of a tile within its frame.

3. The system of claim 1 , wherein the partitioner recognizes a region of interest within the source video, and a size of the region of interest determines the common size of the tiles.

4. The system of claim 1 , wherein the predictive video coder operates according to a standard coding protocol.

5. The system of claim 1 , wherein tiles are persistent across multiple frames of the source video.

6. The system of claim 1 , wherein the predictive video coder comprises:
a video decoder having an input for coded tile data designated as reference tiles and an output for decoded tile data,
a reference tile buffer to store the decoded tile data, and
a predictor.

7. The system of claim 6 , further comprising a controller, wherein
a number of tiles is consistent across a plurality of frames of the video sequence, and
the controller designates each persistence of a tile to serve as a candidate prediction reference at a rate determined by a predetermined policy.

8. The system of claim 1 , further comprising a controller that
estimates complexity of the tiles of the frame from the video source, and
controls operations of the predictive video coder to coding the tiles in a coding order determined by the estimated complexity.

9. The system of coding system of claim 1 , wherein the first frequency is based on an estimated rate of change of image content within the tiles corresponding to the first tile location.

10. A method of coding video data, comprising:
partitioning individual frames of a video source into a plurality of tiles of common size;
coding the frames from the video source according to a video coding protocol including assigning each tile of each frame from the video source its own timestamp according to the video coding protocol, and
coding content of the tiles of the frames with motion compensation according to the video coding protocol having a respective temporal location corresponding to the tiles's respective timestamp and tiles corresponding to a first tile location over a plurality of frames of the video source are designated as a reference tile for the motion compensation at a first frequency lower than tiles corresponding to a second tile location over the plurality of frames; and
outputting the coded tile data to a channel.

11. The method of claim 10 , wherein the first frequency is selected based on a rate of change of image content at the first tile location over the plurality of frames.

Assignees

• Apple Inc

Inventors

• Zhang Dazhong
• Song Peikang
• Wang Beibei
• Gopalan Giribalan
• Keinath Albert E
• Garrido Christopher M
• Conrad David R
• Wu Hsi-Jung
• Jin Ming
• Yuan Hang
• Yang Xiaohua
• Zhou Xiaosong
• Kasarabada Vikrant
• Concion Davide
• Chien Eric L
• Chan Bess C
• Santhanam Karthick
• Chandok Gurtej Singh

Classifications

• H04N19/65

  using error resilience · CPC title

• H04N19/507 Primary

  using conditional replenishment · CPC title

• H04N19/14 Primary

  Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence) · CPC title

• H04N19/132

  Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking · CPC title

• H04N19/174

  the region being a slice, e.g. a line of blocks or a group of blocks · CPC title