Methods and devices for sub-sampling and interleaving multiple images, EG stereoscopic

Patent metadata

Field	Value
Publication number	US-9025670-B2
Application number	US-201013146641-A
Country	US
Kind code	B2
Filing date	Jan 28, 2010
Priority date	Jan 29, 2009
Publication date	May 5, 2015
Grant date	May 5, 2015

Abstract

Sampled data is packaged in checkerboard format for encoding and decoding. The sampled data may be quincunx sampled multi-image video data (e.g., 3D video or a multi-program stream), and the data may also be divided into sub-images of each image which are then multiplexed, or interleaved, in frames of a video stream to be encoded and then decoded using a standardized video encoder. A system for viewing may utilize a standard video decoder and a formatting device that de-interleaves the decoded sub-images of each frame reformats the images for a display device. A 3D video may be encoded using a most advantageous interleaving format such that a preferred quality and compression ratio is reached. In one embodiment, the invention includes a display device that accepts data in multiple formats.

First claim

What is claimed:

1. A computer-implemented method for encoding full resolution multi-view video data, comprising:
sub-sampling the full resolution multi-view video data into a plurality of sampled images including a sampled image of a first view associated with a particular image and a sampled image of a second view associated with the particular image;
separating, via a pattern, the sampled image of the first view and the sampled image of the second view into a plurality of first sub-images and a plurality of second sub-images, respectively, wherein the plurality of first sub-images include (i) a first even row sub-image and a first odd row sub-image, or (ii) a first even column sub-image and a first odd column sub-image, and wherein the plurality of second sub-images include (i) a first even row sub-image and a first odd row sub-image, or (ii) a first even column sub-image and a first odd column sub-image;
packaging at least the plurality of first sub-images and the plurality of second sub-images together into an image frame, wherein the packaging comprises:
interleaving, based on an arrangement format associated with the particular image, the plurality of first sub-images and the plurality of second sub-images to generate an interleaved image; and

encoding the image frame using a video encoder, the image frame including (i) the interleaved image and (ii) data representing the arrangement format associated with the particular image.

2. The method according to claim 1 , wherein the arrangement format provides information of how sub-blocks of sub-images are organized in terms of interleaving.

3. A method, comprising the steps of:
sub-sampling full resolution multi-view video data into sampled images;
separating each of the sampled images into a plurality of sub-images including (i) an even row sub-image and an odd row sub-image, or (ii) an even column sub-image and an odd column sub-image, wherein each sub-image comprises image data of the corresponding sampled image selected via a pattern;
packaging at least the plurality of sub-images of one sampled image together into a single image frame of a video stream, wherein the packaging comprises tiling the sub-images together in one of a variety of arrangements, said arrangements potentially adaptively varying during said video stream and determining desired regions of said sub-images upon which said packaging is performed; and encoding the single image frame via a video encoder, and
encoding an identifier of the arrangement used for tiling the sub-images.

4. The method according to claim 3 , wherein the identifier is a code that is placed inside information of an encoded patterned block and/or that is placed in the image frame.

5. The method according to claim 1 , wherein each sampled image is obtained by sub-sampling only a single view from the multi-view video data.

6. The method according to claim 1 , wherein the step of sub-sampling the full resolution multi-view video data comprises quincunx sampling the full resolution multi-view video data into the plurality of sample images.

7. The method according to claim 6 , wherein the particular image comprises left view image data and right view image data, and wherein the sampled image of the first view and the sampled image of the second view correspond to different views of the particular image in the multi-view video data.

8. The method according to claim 1 , comprising selecting image data of determined rows or columns of the plurality of sampled images.

9. The method according to claim 1 , wherein the step of separating comprises preparing multiple blocks of image data from each of the sampled images.

10. The method according to claim 1 , wherein the arrangement format associated with the particular image is further associated with one or more other images included in the multi-view video data.

11. The method according to claim 1 , wherein an arrangement format associated with one or more images of the multi-view video data adaptively varies based on the one or more images, the sampled images, or the sub-images.

12. The method according to claim 1 , wherein the data representing the arrangement format associated with the particular image is encoded in a portion of the image frame and the packaging of sub-images is performed in other portions of the image frame.

13. The method according to claim 1 , wherein the interleaved image includes (i) a horizontal arrangement of two sub-images of one view, or (ii) a vertical arrangement of two sub-images of one view.

14. The method according to claim 1 , wherein each of the sub-images comprises one or more rectangular blocks, and wherein the blocks of two sub-images are arranged in an interleaved combination.

15. The method according to claim 1 , wherein the packaging maintains at least one dimension equivalent to a dimension of one of the sampled images.

16. The method according to claim 1 , wherein the arrangement format associated with the particular image is selected for enabling advanced scalability features including SNR/resolution scalability, and 2D to 3D scalability.

17. The method according to claim 1 , wherein encoding the image frame comprises encoding the image frame in accordance with one of a JPEG, JPEG-2000, MPEG-2, MPEG-4 AVC, and VC1 encoding standard.

18. A video device, comprising:
a decoder processor configured to:
decode an encoded video signal comprising more than one image per frame in the video signal; and
de-interleave groups of video data interleaved in a frame format in the decoded video signal, wherein the groups of video data comprise one or more groups of video data from a first image and one or more groups of video data from a second image,
wherein de-interleaving the groups of video data comprises de-interleaving the groups of video data from multiple interleaving formats according to one or more identifiers including:
(1) a sub-sampling arrangement map identifier, wherein said sub-sampling arrangement map identifier determines desired regions of sub-images upon which said interleaving is performed, and wherein the sub-sampling arrangement map identifier is encoded as metadata information and made part of an image frame, and
(2) an interleaving identifier that is encoded in the video signal and identifies an interleaving arrangement of a plurality of interleaving arrangements for the sub-images, wherein said interleaving arrangement comprises at least one of a horizontal, vertical, block-based, or map-based interleaving arrangement.

19. The video device according to claim 18 , wherein the decoder processor is further comprising an up converter configured to up convert the de-interleaved groups of data from at least one of the images.

20. The video device according to claim 19 , wherein the up-converted data comprises data of the first image which comprises a first view in a 3D image and data of the second image which comprises a second view of the 3D image.

21. The video device according to claim 18 , wherein the video device is part of at least one of a Blue-Ray DVD player, a media player, a set-top box, a cable box, a computer video card, a tuner, or other electronic device.

22. An encoding system, configured to perform the method of claim 1 .

23. An encoding system, comprising:
a sub-sampler configured to sub-sample images of at least two different views;
a formatter configured to select at least one group of image data from each view and interleave the groups into a singl

Assignees

• Tourapis Alexandros
• Husak Walter J
• Pahalawatta Peshala V
• Leontaris Athanasios
• Dolby Lab Licensing Corp

Inventors

• Tourapis Alexandros
• Husak Walter J
• Pahalawatta Peshala V
• Leontaris Athanasios

Classifications

• H04N13/194

  Transmission of image signals · CPC title

• H04N21/4347 Primary

  Demultiplexing of several video streams · CPC title

• H04N13/161

  Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597) · CPC title

• H04N19/61

  in combination with predictive coding · CPC title

• H04N19/85

  using pre-processing or post-processing specially adapted for video compression · CPC title