Encoding device, encoding method, decoding device, and decoding method for encoding multiple viewpoints for compatibility with existing mode allowing fewer viewpoints

Patent metadata

Field	Value
Publication number	US-9762884-B2
Application number	US-201113816247-A
Country	US
Kind code	B2
Filing date	Aug 31, 2011
Priority date	Sep 3, 2010
Publication date	Sep 12, 2017
Grant date	Sep 12, 2017

Abstract

An encoding device and method, and a decoding device and method, capable of encoding and decoding a multi-viewpoint image in accordance with a mode having compatibility with an existing mode. A compatible encoder generates a compatible stream by encoding an image that is a compatible image. An image converting unit converts the resolution of images that are auxiliary images. An auxiliary encoder generates an encoded stream of the auxiliary image by encoding the auxiliary image of which the resolution is converted. A compatibility information generating unit generates, as compatibility information, information that designates the image as a compatible image. A multiplexing unit transmits the compatible stream, the encoded stream of the auxiliary image, and the compatibility information. The encoding device can encode a 3D image of the multi-viewpoint mode.

First claim

The invention claimed is:

1. An encoding device comprising:
circuitry configured to
generate a first encoded stream by a compatible image, the compatible image being an image that is encoded in an encoding mode for an image of no more than two viewpoints;
generate an auxiliary image used when multi-viewpoint images are generated from the compatible image, the auxiliary image being an image that is used for generating, by interpolating from the compatible image, images of a plurality of viewpoints including more viewpoints than a number of viewpoints represented in the compatible image and the auxiliary image, the auxiliary image being generated by multiplexing images of two of the plurality of viewpoints that are separated by a greatest distance;
convert resolution of the auxiliary image;
generate a second encoded stream by encoding the converted auxiliary image;
set compatible image designating information designating the compatible image; and
transmit the first encoded stream, the second encoded stream, and the compatible image designating information.

2. The encoding device according to claim 1 , wherein the circuitry is further configured to generate a first disparity encoded stream by encoding a compatible disparity image that is a disparity image of the compatible image and generate a second disparity encoded stream by encoding an auxiliary disparity image that is a disparity image of the auxiliary image,
wherein the circuitry is configured to transmit the first disparity encoded stream and the second disparity encoded stream.

3. The encoding device according to claim 2 ,
wherein the circuitry is configured to
convert the resolution of the auxiliary disparity image, and
encode the auxiliary disparity image of which the resolution is converted.

4. The encoding device according to claim 3 , wherein the circuitry is further configured to
multiplex the auxiliary disparity image of which the resolution is converted,
set multiplexing mode information that represents a multiplexing mode of the auxiliary disparity image,
encode the multiplexed auxiliary disparity image, and
transmit the multiplexing mode information.

5. The encoding device according to claim 3 ,
wherein the circuitry is configured to
set conversion mode information that represents a conversion mode of the resolution of the auxiliary disparity image, and
transmit the conversion mode information.

6. The encoding device according to claim 3 , wherein the circuitry is further configured to
multiplex the compatible disparity image in a time direction,
set multiplexing mode information that represents a multiplexing mode of the compatible disparity image,
encode the compatible disparity image multiplexed in the time direction, and
transmit the multiplexing mode information set by the setting unit.

7. The encoding device according to claim 6 ,
wherein circuitry is configured to
multiplex the auxiliary disparity image in a spatial direction,
set the multiplexing mode information of the compatible disparity image and the auxiliary disparity image,
encode the compatible disparity image multiplexed in the time direction and the auxiliary disparity image multiplexed in the spatial direction, and
transmit the multiplexing mode information.

8. The encoding device according to claim 2 , wherein the compatible disparity image is a disparity image that is common to the compatible images.

9. The encoding device according to claim 2 , wherein the circuitry is further configured to
multiplex the auxiliary image and the compatible disparity image in a time direction,
set multiplexing mode information that represents a multiplexing mode,
encode the multiplexed auxiliary image,
encode the multiplexed compatible disparity image, and
transmit the multiplexing mode information.

10. The encoding device according to claim 1 , wherein the circuitry is configured to
set distance information that represents a distance between viewpoints of the compatible image and the auxiliary image, and
transmit the distance information.

11. The encoding device according to claim 1 ,
wherein the circuitry is configured to
multiplex the auxiliary image of which the resolution is converted,
set auxiliary image multiplexing mode information that represents a multiplexing mode of the auxiliary image,
encode the multiplexed auxiliary image, and
transmit the auxiliary image multiplexing mode information.

12. The encoding device according to claim 11 ,
wherein the circuitry is configured to
set auxiliary image resolution converting mode information that represents a conversion mode of the resolution of the auxiliary image, and
transmit the auxiliary image resolution converting mode information.

13. An encoding method to be performed by circuitry of an encoding device, the encoding method comprising:
a compatible image encoding step of generating, by the circuitry, a first encoded stream by a compatible image, the compatible image being an image that is encoded in an encoding mode for an image of no more than two viewpoints;
an auxiliary image generating step of generating, by the circuitry, an auxiliary image used when multi-viewpoint images are generated from the compatible image, the auxiliary image being an image that is used for generating, by interpolating from the compatible image, images of a plurality of viewpoints including more viewpoints than a number of viewpoints of the compatible image and the auxiliary image, the auxiliary image being generated by multiplexing images of two of the plurality of viewpoints that are separated by a greatest distance;
an image converting step of converting, by the circuitry, resolution of the auxiliary image;
an auxiliary image encoding step of generating, by the circuitry, a second encoded stream by encoding the auxiliary image converted in a process of the image converting step;
a setting step of setting, by the circuitry, compatible image designating information designating the compatible image; and
a transmitting step of transmitting, by the circuitry, the first encoded stream generated in a process of the compatible image encoding step, the second encoded stream generated in a process of the auxiliary image encoding step, and the compatible image designating information set in a process of the setting step.

14. A decoding device comprising:
circuitry configured to
receive a first encoded stream that is acquired as a result of encoding a compatible image, the compatible image being an image that is encoded in an encoding mode for an image of no more than two viewpoints, a second encoded stream that is acquired as a result of converting resolution of an auxiliary image used when a multi-viewpoint image is generated from the compatible image and encoding the auxiliary image, the auxiliary image being an image that is used for generating, by interpolating from the compatible image, images of a plurality of viewpoints including more viewpoints than a number of viewpoints of the compatible image and the auxiliary image, the auxiliary image being generated by multiplexing images of two of the plurality of viewpoints that are separated by a greatest distance, and compatible image designating information that designates the compatible image;
identify the first encoded stream based on the compatible image designating information and decode the first encoded stream;
decode the second encoded stream; and
convert the resolution of the auxiliary image, of which the resolution is converted, that is acquired as a result of decoding the second encoded stream.

15. The decoding device accor

Assignees

• Hattori Shinobu
• Takahashi Yoshitomo
• Yonemitsu Jun
• Sony Corp

Inventors

• Hattori Shinobu
• Takahashi Yoshitomo
• Yonemitsu Jun

Classifications

• H04N13/0062

  Electricity · mapped topic

• H04N19/597

  specially adapted for multi-view video sequence encoding · CPC title

• H04N19/70

  characterised by syntax aspects related to video coding, e.g. related to compression standards · CPC title

• H04N19/59

  involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution · CPC title

• H04N13/0271

  Electricity · mapped topic