3D video encoding and decoding methods and apparatus

Patent metadata

Field	Value
Publication number	US-9407902-B1
Application number	US-201213443864-A
Country	US
Kind code	B1
Filing date	Apr 10, 2012
Priority date	Apr 10, 2011
Publication date	Aug 2, 2016
Grant date	Aug 2, 2016

Abstract

Methods and apparatus relating to encoding and decoding stereoscopic (3D) image data, e.g., left and right eye images, are described. Various pre-encoding and post-decoding operations are described in conjunction with difference based encoding and decoding techniques. In some embodiments left and right eye image data is subject to scaling, transform operation(s) and cropping prior to encoding. In addition, in some embodiments decoded left and right eye image data is subject to scaling, transform operations(s) and cropping prior to being output to a display device. Transform information, scaling information and/or cropping information may be included in a bitstream communicating encoded left and right eye images. The amount of scaling can be the same for an entire scene and/or program.

First claim

What is claimed is:

1. A method of processing stereoscopic image data including a left eye image and a right eye image of a first frame pair, comprising:
comparing content of said left eye image and said right eye image of the first frame pair to determine a first image shift which can be used to reduce a difference between the left eye image and right eye image;
performing transform operations on said left eye image and said right eye image to reduce the difference between the left eye image and right eye image, said transform operations including scaling said left and right eye images by a first amount to generate scaled left and right eye images, cropping said scaled left eye image to generate a cropped scaled left eye image including scaled content shifted in a first direction relative to a center of the left eye image, and cropping said scaled right eye image to generate a cropped scaled right eye image including scaled content shifted in a second direction relative to a center of the right eye image;
encoding said cropped scaled left eye image and cropped scaled right eye image to generate a first encoded frame pair, wherein encoding said cropped scaled left eye image and cropped scaled right eye image includes using an encoder which performs difference encoding on at least one of said cropped scaled left and right eye images using the other one of said cropped scaled left and right eye images as a reference image; and
communicating information to a playback device, with said first encoded frame pair, indicating said first amount of scaling.

2. The method of claim 1 , wherein said difference encoding includes motion compensated prediction encoding.

3. The method of claim 1 , wherein said image shift which can be used to reduce the difference between the left eye image and right eye image is a maximum image shift in one of a vertical and horizontal direction required to minimize the difference between the left eye image and the right eye image.

4. The method of claim 1 ,
wherein the left eye image is of a first size; and
wherein cropping said scaled left eye image to generate a cropped scaled left eye image includes:
cropping said scaled left eye image to the first size, said cropped scaled left eye image being the same size as said left eye image.

5. The method of claim 1 ,
wherein said first and second directions are opposite directions; and

wherein said cropped scaled left eye image includes content which is shifted by an equal and opposite amount as compared to content of said cropped scaled right eye image.

6. The method of claim 1 , further comprising:
generating a bitstream communicating said first encoded frame pair, scaling information corresponding to said first encoded frame pair, and shift information corresponding to said first encoded frame pair.

7. The method of claim 6 , further comprising:
receiving a left eye image and a right eye image of a second frame pair, said second frame pair being a frame pair which follows said first frame pair in a sequence of frame pairs;
comparing content of said left eye image and said right eye image of the second frame pair to determine a second image shift which can be used to reduce a difference between the left eye image and right eye image of the second frame pair; and
performing transform operations on said left eye image and said right eye image of the second frame pair to reduce the difference between the left eye image and right eye image of the second frame pair, said transform operations including scaling said left and right eye images of the second frame pair by a second amount to generate scaled left and right eye images, said second amount being determined based on said first amount and said determined second image shift.

8. The method of claim 6 , further comprising:
operating said playback device to:
decode said encoded first frame pair to produce decoded left and right eye images;
scale said decoded left and right eye images by an amount determined from scaling information communicated by the bitstream to generate a scaled decoded left eye image and a scaled decoded right eye image; and
crop said scaled decoded left eye image and scaled decoded right eye image.

9. The method of claim 8
wherein said cropping of said scaled decoded left and right eye images crops said scaled decoded left and right eye images to a size which is the size of said left eye image and right eye image of the first frame pair.

10. The method of claim 9 ,
wherein said scaling performed by said playback device on said decoded left and right eye images is by said first amount; and
wherein said cropping of said scaled decoded left and right eye images produces a cropped decoded left eye image including scaled content shifted relative to a center of the decoded left eye image in a direction opposite the first direction and a cropped decoded right eye image including scaled content shifted relative to a center of the decoded right eye image in a direction opposite the second direction.

11. A method of processing encoded stereoscopic image data including an encoded left eye image and an encoded right eye image of a frame pair, comprising:
decoding said encoded left and right eye images to produce decoded left and right eye images, said encoded left and right eye images having been encoded using difference encoding on at least one of a left eye image and a right eye image using the other one of the left and right eye images as a reference image, said left and right eye images having been produced by transform operations including scaling said left and right eye images to generate scaled left and right eye images, cropping said scaled left eye image to generate a cropped scaled left eye image including scaled content shifted in a first direction and cropping said scaled right eye image to generate a cropped scaled right eye image including scaled content shifted in a second direction, to reduce a difference between the left and right eye images, said second direction being opposite to said first direction;
scaling said decoded left and right eye images by a first amount indicated to a playback device by information supplied to said playback device along with said encoded left eye image and said encoded right eye image or prior to said encoded left and right eye images being supplied to said playback device;
cropping said scaled decoded left and right eye images, said cropping of said scaled decoded left and right eye images producing a cropped scaled decoded left eye image including scaled content shifted relative to the center of the decoded left eye image and a cropped scaled decoded right eye image including scaled content shifted relative to the center of the decoded right eye image; and
outputting said cropped scaled decoded left eye image and right eye image.

12. The method of claim 11 , further comprising:
performing an inverse left eye transform operation on said scaled decoded left eye image prior to cropping said scaled decoded left eye image;
performing an inverse right eye transform operation on said scaled decoded right eye image prior to cropping said scaled decoded right eye image; and
wherein said cropping of said scaled decoded left and right eye images crops said decoded left and right eye images to the size of said encoded left and right eye images.

13. The method of claim 12 ,
wherein said inverse left eye transform operation includes a shift in a direction and amount which is based on left eye transform information supplied with said encoded left eye image;
wherein said inverse right eye transform operation includes a shift in a direction and amount which is b

Assignees

• Cole David Michael
• Moss Alan Mckay
• Nextvr Inc

Inventors

• Cole David Michael
• Moss Alan Mckay

Classifications

• H04N13/04

  Electricity · mapped topic

• H04N13/02 Primary

  Electricity · mapped topic

• H04N19/597 Primary

  specially adapted for multi-view video sequence encoding · CPC title

• H04N13/106

  Processing image signals (for multi-view video sequence encoding H04N19/597) · CPC title

• H04N19/85

  using pre-processing or post-processing specially adapted for video compression · CPC title