Coding and decoding of interleaved image data

Patent metadata

Field	Value
Publication number	US-9877046-B2
Application number	US-201514613651-A
Country	US
Kind code	B2
Filing date	Feb 4, 2015
Priority date	Jan 29, 2009
Publication date	Jan 23, 2018
Grant date	Jan 23, 2018

Abstract

Sampled data is packaged in checkerboard format for encoding and decoding. The sampled data may be quincunx sampled multi-image video data (e.g., 3D video or a multi-program stream), and the data may also be divided into sub-images of each image which are then multiplexed, or interleaved, in frames of a video stream to be encoded and then decoded using a standardized video encoder. A system for viewing may utilize a standard video decoder and a formatting device that de-interleaves the decoded sub-images of each frame reformats the images for a display device. A 3D video may be encoded using a most advantageous interleaving format such that a preferred quality and compression ratio is reached. In one embodiment, the invention includes a display device that accepts data in multiple formats.

First claim

What is claimed:

1. An apparatus comprising a non-transitory computer-readable medium for storing data accessible by a decoder, the computer-readable medium comprising:
a compressed bitstream stored in the computer-readable medium, the compressed bitstream representing:
pixel values of packed video frames, wherein each packed video frame comprises an interleaved arrangement of (i) samples of odd rows and odd columns of a first image, (ii) samples of even rows and even columns of the first image, (iii) samples of odd rows and even columns of a second image, and (iv) samples of even rows and odd columns of the second image, each forming a different portion of the packed video frame, respectively;
first side information indicating a sampling format of the interleaved arrangement wherein the first side information for a particular packed video frame indicates a quincunx sampling format; and
second side information indicating an arrangement of the pixel values of the first and second images in the interleaved arrangement, wherein the second side information for the particular packed video frame indicates a vertical arrangement,
wherein the particular packed video frame comprises (i) samples of the first image arranged as a top portion of the packed video frame, and (ii) samples of the second image arranged as a bottom portion of the packed video frame.

2. An apparatus comprising a non-transitory computer-readable medium for storing data accessible by a decoder, the computer-readable medium comprising:
a compressed bitstream stored in the computer-readable medium, the compressed bitstream representing:
pixel values of packed video frames, wherein each packed video frame comprises an interleaved arrangement of (i) samples of odd rows and odd columns of a first image, (ii) samples of even rows and even columns of the first image, (iii) samples of odd rows and even columns of a second image, and (iv) samples of even rows and odd columns of the second image, each forming a different portion of the packed video frame, respectively;
first side information indicating a sampling format of the interleaved arrangement, wherein the first side information for a particular packed video frame indicates a quincunx sampling format,
second side information indicating an arrangement of the pixel values of the first and second images in the interleaved arrangement, wherein the second side information for the particular packed video frame indicates a vertical arrangement, and
wherein the particular packed video frame comprises (i) samples of odd rows and odd columns of the first image arranged as a top-left portion of the packed video frame, (ii) samples of even rows and even columns of the first image arranged as a top-right portion of the packed video frame, (iii) samples of odd rows and even columns of the second image arranged as a bottom-left portion of the packed video frame, and (iv) samples of even rows and odd columns of the second image arranged as a bottom-right portion of the packed video frame.

3. An apparatus comprising a non-transitory computer-readable medium for storing data accessible by a decoder, the computer-readable medium comprising:
a compressed bitstream stored in the computer-readable medium, the compressed bitstream representing:
pixel values of packed video frames, wherein each packed video frame comprises an interleaved arrangement of (i) samples of odd rows and odd columns of a first image, (ii) samples of even rows and even columns of the first image, (iii) samples of odd rows and even columns of a second image, and (iv) samples of even rows and odd columns of the second image, each forming a different portion of the packed video frame, respectively;
first side information indicating a sampling format of the interleaved arrangement, wherein the first side information for a particular packed video frame indicates a quincunx sampling format,
second side information indicating an arrangement of the pixel values of the first and second images in the interleaved arrangement, wherein the second side information for the particular packed video frame indicates a vertical arrangement, and
wherein the particular packed video frame comprises (i) samples of odd rows and odd columns of the first image arranged as a top-left portion of the packed video frame, (ii) samples of even rows and even columns of the first image arranged as a bottom-right portion of the packed video frame, (iii) samples of odd rows and even columns of the second image arranged as a top-right portion of the packed video frame, and (iv) samples of even rows and odd columns of the second image arranged as a bottom-left portion of the packed video frame.

4. The apparatus of claim 1 , wherein the compressed bitstream comprises a stereoscopic (3D) video signal with a left view and a right view.

5. The apparatus of claim 1 , wherein the first side information or the second side information is changed adaptively in the compressed bitstream.

6. The apparatus of claim 1 , wherein the first side information or the second side information is changed adaptively at one or more of a sequence, picture, and slice level.

7. An apparatus to encode a sequence of multi-view frames into a compressed bitstream, the apparatus comprising:
an input port to receive a first view image and a second view image;
one or more processors configured to:
sample odd rows and odd columns of the first view image to generate a first sub-image of a packed video frame;
sample even rows and even columns of the first view image to generate a second sub-image of the packed video frame;
sample odd rows and even columns of the second view image to generate a third sub-image of the packed video frame;
sample even rows and odd columns of the second view image to generate a fourth sub-image of the packed video frame;
generate first side information for the packed video frame indicating a quincunx sampling format for the interleaved first and second view images;
generate second side information for the packed video frame indicating how the first, second, third, and fourth sub-images are tiled together to form the packed video frame, wherein the second side information indicates that (i) the first sub-image forms a top-left portion of the packed video frame, (ii) the second sub-image forms a bottom-left portion of the packed video frame, (iii) the third sub-image forms a top-right portion of the packed video frame, and (iv) the fourth sub-image forms a bottom-right portion of the packed video frame; and

an output port to output the packed video frame, the first side information, and the second side information.

8. An apparatus to encode a sequence of multi-view frames into a compressed bitstream, the apparatus comprising:
an input port to receive a first view image and a second view image;
one or more processors configured to:
sample odd rows and odd columns of the first view image to generate a first sub-image of a packed video frame;
sample even rows and even columns of the first view image to generate a second sub-image of the packed video frame;
sample odd rows and even columns of the second view image to generate a third sub-image of the packed video frame;
sample even rows and odd columns of the second view image to generate a fourth sub-image of the packed video frame;
generate first side information for the packed video frame indicating a quincunx sampling format for the interleaved first and second view images;
generate second side information for the packed video frame indicating how the first, second, third, and fourth sub-images are tiled together to form the packed video frame, wherein the second side information indicates that (i) the first sub-image forms a top-left portion of the packed video fr

Assignees

• Dolby Laboratories Licensing Corp

Inventors

• Tourapis Alexandros
• Husak Walter J
• Pahalawatta Peshala V
• Leontaris Athanasios

Classifications

• H04N19/16

  for a given display mode, e.g. for interlaced or progressive display mode · CPC title

• H04N21/4382

  Demodulation or channel decoding, e.g. QPSK demodulation (analog front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002) · CPC title

• H04N19/85

  using pre-processing or post-processing specially adapted for video compression · CPC title

• H04N21/2365 Primary

  Multiplexing of several video streams · CPC title

• H04N19/60

  using transform coding · CPC title