Picture encoding method, picture decoding method, picture encoding apparatus, picture decoding apparatus, picture encoding program, picture decoding program and recording medium

Patent metadata

Field	Value
Publication number	US-2015245062-A1
Application number	US-201314430492-A
Country	US
Kind code	A1
Filing date	Sep 24, 2013
Priority date	Sep 25, 2012
Publication date	Aug 27, 2015
Grant date	—

Abstract

An picture encoding method includes steps of converting a reference depth map into a virtual depth map that is a depth map of an object photographed in a target picture, generating a depth value of an occlusion region in which there is no depth value in the reference depth map generated by an anteroposterior relationship of the object by assigning a depth value of which a correspondence relationship with a region on the same object as an object shielded in the reference picture is obtained to the occlusion region, and performing picture prediction between views by generating a disparity-compensated picture for the encoding target picture from the virtual depth map and the reference picture after the depth value of the occlusion region is generated.

First claim

1 . A picture encoding method for performing encoding a multiview picture which includes pictures for a plurality of views while predicting a picture between the views using an encoded reference picture for a view different from a view of an encoding target picture and a reference depth map that is a depth map of an object in the reference picture, the method comprising:
a depth map conversion step of converting the reference depth map into a virtual depth map that is a depth map of the object in the encoding target picture; an occlusion region depth generation step of generating a depth value of an occlusion region in which there is no depth value assigned in the reference depth map generated by an anteroposterior relationship of the object by assigning a depth value of which a correspondence relationship with a region on the same object as the object shielded in the reference picture is obtained to the occlusion region; and an inter-view picture prediction step of performing picture prediction between the views by generating a disparity-compensated picture for the encoding target picture from the virtual depth map and the reference picture after the depth value of the occlusion region is generated.

2 . The picture encoding method according to claim 1 ,
wherein the occlusion region depth generation step includes generating the depth value of the occlusion region on an assumption of continuity of an object shielding the occlusion region on the reference depth map.

3 . The picture encoding method according to claim 1 , further comprising:
an occlusion generation pixel border determination step of determining a pixel border on the reference depth map corresponding to the occlusion region, wherein the occlusion region depth generation step includes generating the depth value of the occlusion region by converting a depth of an assumed object into a depth on the encoding target picture on an assumption that an object continuously exists from the same depth value as a depth value of a pixel having a depth value indicating proximity to the view to the same depth value as a depth value of a pixel having a depth value indicating distance from the view in a position of the pixel having a depth value indicating proximity to the view on the reference depth map for each set of pixels of the reference depth map adjacent to the occlusion generation pixel border.

4 . The picture encoding method according to claim 1 , further comprising:
an object region determination step of determining an object region on the virtual depth map for a region shielding the occlusion region on the reference depth map; and an object region extension step of extending a pixel in a direction of the occlusion region in the object region, wherein the occlusion region depth generation step includes generating the depth value of the occlusion region by smoothly interpolating the depth value between a pixel generated through the extension and a pixel adjacent to the occlusion region and present in an opposite direction from the object region.

5 . The picture encoding method according to claim 1 ,
wherein the depth map conversion step includes obtaining a corresponding pixel on the virtual depth map for each reference pixel of the reference depth map and performing conversion to a virtual depth map by assigning a depth indicating the same three-dimensional position as the depth for the reference pixel to the corresponding pixel.

6 . A picture decoding method for performing decoding a decoding target picture of a multiview picture while predicting a picture between views using a decoded reference picture and a reference depth map that is a depth map of an object in the reference picture, the method comprising:
a depth map conversion step of converting the reference depth map into a virtual depth map that is a depth map of the object in the decoding target picture; an occlusion region depth generation step of generating a depth value of an occlusion region in which there is no depth value assigned in the reference depth map generated by an anteroposterior relationship of the object by assigning a depth value of which a correspondence relationship with a region on the same object as the object shielded in the reference picture is obtained to the occlusion region; and an inter-view picture prediction step of performing picture prediction between the views by generating a disparity-compensated picture for the decoding target picture from the virtual depth map and the reference picture after the depth value of the occlusion region is generated.

7 . The picture decoding method according to claim 6 ,
wherein the occlusion region depth generation step includes generating the depth value of the occlusion region on an assumption of continuity of an object shielding the occlusion region on the reference depth map.

8 . The picture decoding method according to claim 6 , further comprising:
an occlusion generation pixel border determination step of determining a pixel border on the reference depth map corresponding to the occlusion region, wherein the occlusion region depth generation step includes generating the depth value of the occlusion region by converting a depth of an assumed object into a depth on the encoding target picture on an assumption that an object continuously exists from the same depth value as a depth value of a pixel having a depth value indicating proximity to the view to the same depth value as a depth value of a pixel having a depth value indicating distance from the view in a position of the pixel having a depth value indicating proximity to the view on the reference depth map for each set of pixels of the reference depth map adjacent to the occlusion generation pixel border.

9 . The picture decoding method according to claim 6 , further comprising:
an object region determination step of determining an object region on the virtual depth map for a region shielding the occlusion region on the reference depth map; and an object region extension step of extending a pixel in a direction of the occlusion region in the object region, wherein the occlusion region depth generation step includes generating the depth value of the occlusion region by smoothly interpolating the depth value between a pixel generated through the extension and a pixel adjacent to the occlusion region and present in an opposite direction from the object region.

10 . The picture decoding method according to claim 6 ,
wherein the depth map conversion step includes obtaining a corresponding pixel on the virtual depth map for each reference pixel of the reference depth map and performing conversion to a virtual depth map by assigning a depth indicating the same three-dimensional position as the depth for the reference pixel to the corresponding pixel.

11 . A picture encoding apparatus for performing encoding a multiview picture which includes pictures for a plurality of views while predicting a picture between the views using an encoded reference picture for a view different from a view of an encoding target picture and a reference depth map that is a depth map of an object in the reference picture, the apparatus comprising:
a depth map conversion unit that converts the reference depth map into a virtual depth map that is a depth map of the object in the encoding target picture; an occlusion region depth generation unit that generates a depth value of an occlusion region in which there is no depth value assigned in the reference depth map generated by an anteroposterior relationship of the object by assigning a depth value of which a correspondence relationship with a region on the same object as the object shielded in the reference picture is

Assignees

• Nippon Telegraph & Telephone

Inventors

• Shimizu Shinya
• Sugimoto Shiori
• Kimata Hideaki
• Kojima Akira

Classifications

• H04N19/52

  by predictive encoding · CPC title

• H04N19/577

  Motion compensation with bidirectional frame interpolation, i.e. using B-pictures · CPC title

• H04N19/597 Primary

  specially adapted for multi-view video sequence encoding · CPC title

• H04N13/161

  Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597) · CPC title

• H04N13/111

  Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation · CPC title