Processing stereo images

Patent metadata

Field	Value
Publication number	US-9519956-B2
Application number	US-201414193508-A
Country	US
Kind code	B2
Filing date	Feb 28, 2014
Priority date	Feb 28, 2014
Publication date	Dec 13, 2016
Grant date	Dec 13, 2016

Abstract

A method comprises estimating disparity values for pixels located in an occluded region of a first one of a defocused pair of stereo images using: a first set of blur map data corresponding to the first one of the defocused pair of stereo images; a first set of disparity map data corresponding to the first one of the defocused pair of stereo images; and a first set of occlusion map data corresponding to the first one of the defocused pair of stereo images.

First claim

The invention claimed is:

1. A method comprising:
estimating disparity values for pixels located in an occluded region of a first one of a defocused pair of stereo images using:
a first set of blur map data corresponding to the first one of the defocused pair of stereo images;
a first set of disparity map data corresponding to the first one of the defocused pair of stereo images; and
a first set of occlusion map data corresponding to the first one of the defocused pair of stereo images; and

generating a first set of blur-disparity map data corresponding to the first one of the defocused pair of stereo images, the first set of blur-disparity map data being generated based on:
the first set of disparity map data;
the first set of occlusion map data; and
the first set of blur map data,

wherein each pixel in the first set of blur-disparity map data has an associated mean blur value, and wherein estimating the disparity values for the pixels located in the occluded region of the first one of the defocused pair of stereo images comprises:
using the first set of occlusion map data to identify at least one occluded pixel in the first set of blur-disparity map data; and
for each occluded pixel in the first set of blur-disparity map data:
calculating a combined difference between the mean blur value of the occluded pixel and the mean blur value of each non-occluded pixel in the first set of blur-disparity map data within an M×N neighbourhood of the occluded pixel;
for each mean blur value within the M×N neighbourhood, calculating the mean blur difference; and
identifying the mean blur value having the lowest mean blur difference, wherein the estimated disparity value for the occluded pixel is the disparity value which corresponds to the mean blur value having the lowest mean blur difference.

2. The method of claim 1 comprising updating the first set of disparity map data with the estimated disparity values to generate a first set of updated disparity map data.

3. The method of claim 2 , comprising:
applying at least one filter to the first set of updated disparity map data.

4. The method of claim 3 , comprising:
applying a first filter to the first set of updated disparity map data; and
applying a second filter to the first-filtered first set of updated disparity map data.

5. The method of claim 4 , wherein the first filter is one of a morphological erosion filter and a morphological dilation filter and wherein the second filter is the other of the morphological erosion filter and the morphological dilation filter.

6. The method of claim 1 , wherein generating the first set of blur-disparity map data comprises:
using the first set of occlusion map data to identify pixels in the first set of disparity map data which correspond to the non-occluded pixels in the first one of the defocused pair of stereo images, each identified pixel in the set of disparity map data having a disparity value;
for each different disparity value of the identified pixels:
identifying a set of non-occluded pixels in the first set of blur map data, the set of non-occluded pixels in the first set of blur map data corresponding to the identified pixels in the first set of disparity map data which have the different disparity value, each identified non-occluded pixel in the first set of blur map data having a blur value;
calculating the mean blur value of the set of non-occluded pixels in the first set of blur map data; and
assigning the calculated mean blur value to the pixels in the first set of blur-disparity map data which correspond to the identified set of non-occluded pixels in the first set of blur map data.

7. The method of claim 6 , comprising, if none of the previously-calculated mean blur values have a contributing pixel having the blur value of an identified occluded pixel in the first set of blur map data, assigning to the occluded pixel in the first set of blur-disparity map data, which corresponds to the identified occluded pixel, the mean blur value which is closest to the blur value of the identified occluded pixel.

8. The method of claim 1 , wherein the combined difference is a weighted average of the absolute difference between the mean blur value of the occluded pixel and the mean blur value of the non-occluded pixel and the geometric Euclidian distance between occluded pixel and the non-occluded pixel.

9. The method of claim 1 , wherein the mean blur difference for each mean blur value within the M×N neighbourhood is calculated by summing the combined differences of all the non-occluded pixels having the mean blur value and normalising the result by the total number of the non-occluded pixels having the mean blur value.

10. The method of claim 1 , comprising:
generating the first set of disparity map data using the pair of stereo images.

11. The method of claim 1 comprising:
generating the first set of occlusion map data using the first one of the defocused pair of stereo images and the first set of disparity map data.

12. The method of claim 1 , comprising:
generating the first set of blur map data using the first one of the defocused pair of stereo images.

13. The method of claim 1 comprising:
estimating disparity values for pixels located in an occluded region of a second one of a defocused pair of stereo images using:
a second set of blur map data corresponding to the second one of the defocused pair of stereo images;
a second set of disparity map data corresponding to the second one of the defocused pair of stereo images; and
a second set of occlusion map data corresponding to the second one of the defocused pair of stereo images.

14. The method of claim 13 comprising:
generating a second set of blur-disparity map data corresponding to the second one of the defocused pair of stereo images, the second set of blur-disparity map data being generated based on:
the second set of disparity map data;
the second set of occlusion map data; and
the second set of blur map data;

estimating the disparity values for the pixels located in the occluded region of the second one of the defocused pair of stereo images using the second set of occlusion map data and the second set of blur-disparity map data.

15. The method of claim 13 comprising updating the second set of disparity map data with the estimated disparity values for pixels located in the occluded region of the second one of the defocused pair of stereo images, to generate a second set of updated disparity map data.

16. The method of claim 15 , comprising:
applying at least one filter to the second set of updated disparity map data.

17. The method of claim 16 , comprising:
applying a first filter to the second set of updated disparity map data; and
applying a second filter to the first-filtered second set of updated disparity map data.

18. The method of claim 17 , wherein the first filter is one of a morphological erosion filter and a morphological dilation filter and wherein the second filter is the other of the morphological erosion filter and the morphological dilation filter.

19. An apparatus comprising:
at least one processor; and
at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to:
estimate disparity values for pixels located in an occluded region of a first one of a defocused pair of stereo images using:
a first set of blur map data corresponding to the first one of the defocused pair of stereo i

Assignees

• Nokia Technologies Oy

Inventors

• Chen Ching-Hui
• Zhou Hui
• Ahonen Timo

Classifications

• G06T5/003 Primary

  Physics · mapped topic

• G06T2207/20228

  Disparity calculation for image-based rendering · CPC title

• G06T2207/10012

  Stereo images · CPC title

• G06T7/0075

  Physics · mapped topic

• G06T7/593 Primary

  from stereo images · CPC title