Image processing device and image processing method

Patent metadata

Field	Value
Publication number	US-10122986-B2
Application number	US-201213981650-A
Country	US
Kind code	B2
Filing date	Feb 16, 2012
Priority date	Feb 18, 2011
Publication date	Nov 6, 2018
Grant date	Nov 6, 2018

Abstract

The present technique relates to an image processing device and an image processing method for realizing accurate depth image warping operations at the receiving side when depth images with reduced resolutions are transmitted. A parallax image generating unit reduces the resolutions of parallax images B 1 ′ and C 1 ′. A boundary information generating unit generates boundary information about the parallax images B 1 ′ and C 1 ′. A multiplexing unit transmits a multiplexed stream containing the parallax images B 1 ′ and C 1 ′ with the reduced resolutions and the boundary information. The present technique can be applied to encoding devices that encode glasses-free 3D images, for example.

First claim

The invention claimed is:

1. An image processing device, comprising:
at least one processor configured to:
reduce a resolution of a depth image;
detect a boundary position on the depth image, where pixel values in the depth image vary, based on a pixel value difference that is larger than a threshold value,
wherein the pixel value difference is between a plurality of first adjacent pixels in a horizontal direction in the depth image;
generate boundary information on a pixel basis, wherein the boundary information indicates the plurality of first adjacent pixels adjacent to the boundary position, based on the boundary position;
generate left-side boundary information on the pixel basis, wherein the left-side boundary information indicates a plurality of second adjacent pixels located on a left side of the boundary position among the plurality of first adjacent pixels indicated by the boundary information,
wherein, in an increased resolution depth image derived from the depth image having reduced resolution, a first pixel value of a plurality of first pixels adjacent to a plurality of second pixels is changed to a second pixel value of the plurality of second pixels, and wherein the first pixel value is changed to the second pixel value based on the boundary information and the left-side boundary information; and

transmit the depth image having the reduced resolution, the boundary information, and the left-side boundary information.

2. The image processing device according to claim 1 , wherein
the at least one processor are further configured to transmit the depth image having the reduced resolution, the boundary information, and resolution conversion information.

3. The image processing device according to claim 1 , wherein
the at least one processor is further configured to:
encode the depth image, having the reduced resolution, on a block basis,
generate the boundary information on one of the pixel basis or on the block basis, and
transmit the encoded depth image, and the generated boundary information.

4. The image processing device according to claim 1 , wherein the at least one processor is further configured to transmit a viewpoint image, the depth image having the reduced resolution, and the boundary information.

5. The image processing device according to claim 1 , wherein the at least one processor is further configured to set the generated boundary information to “1” based on the pixel value difference that is larger than the threshold value.

6. The image processing device according to claim 1 , wherein the at least one processor is further configured to set the generated boundary information to “0” based on the pixel value difference that is smaller than the threshold value.

7. The image processing device according to claim 1 , wherein a plurality of pixels adjacent to the boundary position include at least one pattern, wherein the at least one pattern is different for the plurality of first adjacent pixels.

8. The image processing device according to claim 7 , wherein the at least one processor is further configured to detect the boundary position based on a difference of the at least one pattern for the plurality of first adjacent pixels.

9. An image processing method, comprising:
in an image processing device:
reducing a resolution of a depth image;
detecting a boundary position on the depth image, where pixel values in the depth image vary, based on a pixel value difference that is larger than a threshold value,
wherein the pixel value difference is between a plurality of first adjacent pixels in a horizontal direction of the depth image;
generating boundary information on a pixel basis, wherein the boundary information indicates the plurality of first adjacent pixels adjacent to the boundary position, based on the boundary position;
generating left-side boundary information on the pixel basis, wherein the left-side boundary information indicates a plurality of second adjacent pixels located on a left side of the boundary position among the plurality of first adjacent pixels indicated by the boundary information,
wherein, in an increased resolution depth image derived from the depth image having reduced resolution, a first pixel value of a plurality of first pixels adjacent to a plurality of second pixels is changed to a second pixel value of the plurality of second pixels, and wherein the first pixel value is changed to the second pixel value based on the boundary information and the left-side boundary information; and

transmitting the depth image having the reduced resolution, the generated boundary information, and the left-side boundary information.

10. An image processing device, comprising:
at least one processor configured to:
receive a first depth image having a reduced resolution, and boundary information which indicates a plurality of first pixels adjacent to a boundary position where a plurality of pixel values in the first depth image vary;
increase resolution of the first depth image;
generate a second depth image of a virtual viewpoint based on the first depth image having the increased resolution, the boundary information, and a position of the virtual viewpoint; and

based on the boundary information, change a first pixel value of the plurality of first pixels indicated by the boundary information to a second pixel value of a plurality of second pixels adjacent to the plurality of first pixels in the first depth image having the increased resolution.

11. The image processing device according to claim 10 , wherein
the at least one processor is further configured to:
receive the first depth image having the reduced resolution, the boundary information on a pixel basis, and left-side boundary information that indicates a plurality of third pixels located on a left side of the boundary position among the plurality of first pixels indicated by the boundary information, and
based on the boundary information, the left-side boundary information, and the position of the virtual viewpoint, process the first depth image having the increased resolution.

12. The image processing device according to claim 11 , wherein, based on the boundary information and the left-side boundary information, the at least one processor is further configured to change a third pixel value of the plurality of first pixels indicated by the boundary information and the left-side boundary information to a fourth pixel value of left-side pixels adjacent to the plurality of first pixels in the first depth image having the increased resolution.

13. The image processing device according to claim 12 , wherein the at least one processor is further configured to change a fifth pixel value of a plurality of fourth pixels different from the first pixel value of the plurality of first pixels indicated by the boundary information and the left-side boundary information to a sixth pixel value of right-side pixels adjacent to the plurality of first pixels.

14. The image processing device according to claim 10 , wherein
the at least one processor is further configured to:
receive the first depth image having the reduced resolution, the boundary information, and resolution conversion information for the reduced resolution, and
increase resolution of the received first depth image based on the resolution conversion information.

15. The image processing device according to claim 14 , wherein the at least one processor is further configured to increase resolution of the first depth image based on interpolation of a seventh pixel value of each pixel in the first depth image having the increased reso

Assignees

• Hattori Shinobu
• Takahashi Yoshitomo
• Sony Corp

Inventors

• Hattori Shinobu
• Takahashi Yoshitomo

Classifications

• H04N13/161

  Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597) · CPC title

• H04N13/106

  Processing image signals (for multi-view video sequence encoding H04N19/597) · CPC title

• H04N13/111 Primary

  Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation · CPC title

• G06T7/13

  Edge detection · CPC title

• H04N2213/005

  Aspects relating to the "3D+depth" image format · CPC title