Image processing apparatus and image processing method

Patent metadata

Field	Value
Publication number	US-9615079-B2
Application number	US-201214003941-A
Country	US
Kind code	B2
Filing date	Mar 9, 2012
Priority date	Mar 18, 2011
Publication date	Apr 4, 2017
Grant date	Apr 4, 2017

Abstract

The present technique relates to an image processing apparatus and an image processing method capable of generating a color image of a display viewpoint using a color image and a depth image of a predetermined viewpoint.
The viewpoint generation information generation unit generates viewpoint generation information used to generate a color image of a display viewpoint in accordance with a generation method of the color image of the display viewpoint obtained by performing warping processing using multi-viewpoint corrected color images and multi-viewpoint depth images. The multi-viewpoint image encoding unit encodes the multi-viewpoint corrected color images and the multi-viewpoint depth images, and transmits them with the viewpoint generation information. The present technique can be applied to, for example, a multi-viewpoint image processing apparatus.

First claim

The invention claimed is:

1. An image processing apparatus comprising:
an encoding unit for generating a bit stream by encoding a color image of a viewpoint and a depth image of the viewpoint;
a generation unit for generating viewpoint generation information used to generate a color image of a display viewpoint, in accordance with a generation method of the color image of the display viewpoint obtained by performing warping processing using the color image and the depth image; and
a transmission unit for transmitting the bit stream generated by the encoding unit and the viewpoint generation information generated by the generation unit,
wherein:
the depth image is a depthwise image including a depth value representing a position of a subject in a depth direction at each pixel of the color image, or a parallax image including a parallax value representing a distance between each pixel of the color image and a pixel of a color image at a base point corresponding to the pixel,
the generation unit generates, as the viewpoint generation information, depth image identification indicating that the depth image is the depthwise image or the parallax image,
when the depth image is the depthwise image, the generation unit generates, as the viewpoint generation information, depthwise minimum value information indicating a minimum value of world coordinate value at a position in the depth direction in the depthwise image and depthwise maximum value information indicating a maximum value of world coordinate value at a position in the depth direction in the depthwise image, and
when the depth image is the parallax image, the generation unit generates, as the viewpoint generation information, parallax minimum value information indicating a minimum value of the distance of world coordinate value in the parallax image and parallax maximum value information indicating a maximum value of the distance of world coordinate value in the parallax image.

2. The image processing apparatus according to claim 1 , wherein the transmission unit transmits the viewpoint generation information as auxiliary information of the bit stream.

3. The image processing apparatus according to claim 1 , wherein the transmission unit transmits the viewpoint generation information as SEI (Supplemental Enhancement Information) of the bit stream.

4. The image processing apparatus according to claim 1 , wherein the generation unit generates, as the viewpoint generation information, information for identifying the color image or information for identifying the depth image.

5. The image processing apparatus according to claim 1 , wherein:
the depth image is a parallax image including a parallax value representing a distance between each pixel of the color image and a pixel of a color image at a base point corresponding to the pixel,
the generation unit generates, as the viewpoint generation information, parallax minimum value information indicating a minimum value of the parallax value of the parallax image and parallax maximum value information indicating a maximum value of the parallax value of the parallax image, and
the transmission unit transmits the viewpoint generation information as SEI (Supplemental Enhancement Information).

6. The image processing apparatus according to claim 1 , wherein
the depth image is a depthwise image including a depth value representing a position of a subject in a depth direction at each pixel of the color image,
the generation unit generates, as the viewpoint generation information, depthwise minimum value information indicating a minimum value of a pixel value of the depth image and depthwise maximum value information indicating a maximum value of the pixel value of the depth image, and
the transmission unit transmits the viewpoint generation information as SEI (Supplemental Enhancement Information).

7. The image processing apparatus according to claim 1 , wherein:
the generation unit generates, as the viewpoint generation information, depth minimum value information indicating a minimum value of a pixel value in the depth image, depth maximum value information indicating a maximum value of a pixel value in the depth image, and image-capturing position distance information indicating a distance between image-capturing positions of color images corresponding to depth images, and
the transmission unit transmits the viewpoint generation information as SEI (Supplemental Enhancement Information).

8. An image processing method for an image processing apparatus, comprising:
an encoding step for generating a bit stream by encoding a color image of a viewpoint and a depth image of the viewpoint;
a generation step for generating viewpoint generation information used to generate a color image of a display viewpoint, in accordance with a generation method of the color image of the display viewpoint obtained by performing warping processing using the color image and the depth image; and
a transmission step for transmitting the bit stream generated in the processing of the encoding step and the viewpoint generation information generated in the processing of the generation step,
wherein:
the depth image is a depthwise image including a depth value representing a position of a subject in a depth direction at each pixel of the color image, or a parallax image including a parallax value representing a distance between each pixel of the color image and a pixel of a color image at a base point corresponding to the pixel,
the generation step comprises generating, as the viewpoint generation information, depth image identification indicating that the depth image is the depthwise image or the parallax image,
when the depth image is the depthwise image, the generation step comprises generating, as the viewpoint generation information, depthwise minimum value information indicating a minimum value of world coordinate value at a position in the depth direction in the depthwise image and depthwise maximum value information indicating a maximum value of world coordinate value at a position in the depth direction in the depthwise image, and
when the depth image is the parallax image, the generation step comprises generating, as the viewpoint generation information, parallax minimum value information indicating a minimum value of the distance of world coordinate value in the parallax image and parallax maximum value information indicating a maximum value of the distance of world coordinate value in the parallax image.

9. The image processing apparatus according to claim 8 ,
wherein the viewpoint generation information includes image-capturing position distance information indicating a distance between image-capturing positions of color images corresponding to depth images.

Assignees

• Takahashi Yoshitomo
• Hattori Shinobu
• Sakurai Hironari
• Sony Corp

Inventors

• Takahashi Yoshitomo
• Hattori Shinobu
• Sakurai Hironari

Classifications

• H04N19/597 Primary

  specially adapted for multi-view video sequence encoding · CPC title

• H04N13/15

  for colour aspects of image signals · CPC title

• G06T7/60

  Analysis of geometric attributes · CPC title

• H04N13/349

  Multi-view displays for displaying three or more geometrical viewpoints without viewer tracking (for viewing without the aid of special glasses using fly-eye lenses H04N13/307) · CPC title

• H04N13/128

  Adjusting depth or disparity · CPC title