Image processing device and image processing method

Patent metadata

Field	Value
Publication number	US-10460422-B2
Application number	US-201615765086-A
Country	US
Kind code	B2
Filing date	Sep 9, 2016
Priority date	Nov 10, 2015
Publication date	Oct 29, 2019
Grant date	Oct 29, 2019

Abstract

The polarization imaging unit generates a polarized image including pixels for each of a plurality of polarization components. The demosaicing unit calculates a pixel signal for each polarization component by using the pixel signal of the target pixel of the polarized image and the pixel signal of the pixel for each of the identical polarization components located near the target pixel. In one example, a low frequency component is calculated for each polarization component using the pixel signal of the pixel located near the target pixel for each of the identical polarization components. In addition, component information indicating relationship between the low frequency component of the polarization component of the polarized image and the pixel signal of the target pixel is acquired. Furthermore, the pixel signal for each polarization component in the target pixel is calculated based on the low frequency component and the component information for each polarization component.

First claim

The invention claimed is:

1. An image processing device comprising:
a demosaicing unit configured to use a pixel signal of a target pixel of a polarized image including pixels for each of a plurality of polarization components and a pixel signal of a pixel for each of identical polarization components located near the target pixel to calculate a pixel signal for each polarization component different from the polarization component of the polarized image in the target pixel,
wherein the demosaicing unit calculates the pixel signal for each polarization component in the target pixel on a basis of a low frequency component for each polarization component and component information indicating a relationship between the low frequency component of the polarization component of the polarized image and the pixel signal of the target pixel, the low frequency component for each polarization component being calculated by using the pixel signal of the pixel located near the target pixel for each of the identical polarization components, and
wherein the demosaicing unit is implemented via at least one processor.

2. The image processing device according to claim 1 , further comprising:
a low frequency component calculation unit configured to calculate the low frequency component by performing two-dimensional filtering using the pixel signal of the pixel of the identical polarization component located near the target pixel,
wherein the low frequency component calculation unit is implemented via at least one processor.

3. The image processing device according to claim 1 ,
wherein the demosaicing unit generates, by using each pixel of the polarized image as the target pixel, a polarized image for each polarization component.

4. An image processing device comprising:
a demosaicinq unit configured to use a pixel signal of a target pixel of a polarized image including pixels for each of a plurality of polarization components and a pixel signal of a pixel for each of identical polarization components located near the target pixel to calculate a pixel signal for each polarization component different from the polarization component of the polarized image in the target pixel; and
a normal information acquisition unit configured to acquire normal information by using the pixel signal for each of the plurality of polarization components that is calculated by the demosaicing,
wherein the demosaicinq unit and the normal information acquisition unit are each implemented via at least one processor.

5. The image processing device according to claim 4 ,
wherein the normal information acquisition unit performs fitting to a polarization characteristic model by using the pixel signal for each of the plurality of polarization components to acquire the normal information from the fitted polarization characteristic model.

6. The image processing device according to claim 4 ,
wherein the demosaicing unit generates, by using each pixel of the polarized image as the target pixel, a polarized image for each polarization component.

7. An image processing device comprising:
a demosaicing unit configured to use a pixel signal of a target pixel of a polarized image including pixels for each of a plurality of polarization components and a pixel signal of a pixel for each of identical polarization components located near the target pixel to calculate a pixel signal for each polarization component different from the polarization component of the polarized image in the target pixel; and
a normal information acquisition unit configured to acquire normal information by using the pixel signal of the target pixel calculated by the demosaicing unit,
wherein the polarized image including the pixels for each of the plurality of polarization components is a color polarized image,
wherein the demosaicing unit uses the pixel signal of the pixel for each of the identical polarization components located near the target pixel for each color component to calculate the pixel signal for each polarization component in the target pixel for each color component, and
wherein the demosaicing unit and the normal information acquisition unit are each implemented via at least one processor.

8. The image processing device according to claim 7 ,
wherein the normal information acquisition unit calculates luminance for each polarization component by using the pixel signal for each color component that is calculated by the demosaicing unit for each polarization component to acquire the normal information on a basis of the calculated luminance for each polarization component.

9. The image processing device according to claim 7 , further comprising:
a white balance adjustment unit configured to adjust white balance of the color polarized image,
wherein the demosaicing unit uses the color polarized image subjected to the white balance adjustment to calculate the pixel signal for each of the plurality of polarization components for each color component,
wherein the normal information acquisition unit calculates color difference for each polarization component by using the pixel signal for each color component that is calculated by the demosaicing unit for each polarization component to acquire the normal information on a basis of the calculated color difference for each polarization component, and
wherein the white balance adjustment unit is implemented via at least one processor.

10. The image processing device according to claim 7 ,
wherein the demosaicing unit generates, by using each pixel of the polarized image as the target pixel, a polarized image for each polarization component.

11. An image processing device comprising:
a demosaicinq unit configured to use a pixel signal of a target pixel of a polarized image including pixels for each of a plurality of polarization components and a pixel signal of a pixel for each of identical polarization components located near the target pixel to calculate a pixel signal for each polarization component different from the polarization component of the polarized image in the target pixel; and
a non-polarized image acquisition unit configured to generate a pixel signal of a non-polarized image from the pixel signal for each of the plurality of polarization components,
wherein the demosaicinq unit and the non-polarized image acquisition unit are each implemented via at least one processor.

12. The image processing device according to claim 11 ,
wherein the non-polarized image acquisition unit calculates the pixel signal of the non-polarized image by averaging the pixel signals for each of the plurality of polarization components.

13. The image processing device according to claim 11 ,
wherein the polarized image including the pixels for each of the plurality of polarization components is a color polarized image,
wherein the demosaicing unit uses the pixel signal of the pixel for each of the identical polarization components located near the target pixel for each color component to calculate the pixel signal for each polarization component in the target pixel for each color component, and
wherein the non-polarized image acquisition unit calculates the pixel signal of the non-polarized image for each color component by using the pixel signal for each polarization component that is calculated by the demosaicing unit for each color component.

14. The image processing device according to claim 11 ,
wherein the demosaicing unit generates, by using each pixel of the polarized image as the target pixel, a polarized image for each polarization component.

15. An image processing device comprising:
a demosaicing unit configured

Assignees

• Sony Corp

Inventors

• Kurita Teppei
• Kaizu Shun
• Kondo Yuhi
• Hirasawa Yasutaka
• Lu Ying

Classifications

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• H04N9/03

  Circuitry for demodulating colour component signals modulated spatially by colour striped filters by frequency separation · CPC title

• H04N25/13

  characterised by the spectral characteristics of the filter elements · CPC title

• H04N23/10

  for generating image signals from different wavelengths · CPC title

• G01J4/02 Primary

  Polarimeters of separated-field type; Polarimeters of half-shadow type · CPC title