Imaging apparatus, image processing apparatus, and image processing method

Patent metadata

Field	Value
Publication number	US-11997398-B2
Application number	US-202017602588-A
Country	US
Kind code	B2
Filing date	Feb 7, 2020
Priority date	Apr 19, 2019
Publication date	May 28, 2024
Grant date	May 28, 2024

Abstract

An imaging apparatus (10) includes an imaging unit (11) that has a first pixel block and a second pixel block provided in a pixel region, the first pixel block including polarization pixels with two different polarization directions and a non-polarization pixel, the second pixel block including polarization pixels with a plurality of polarization directions including a different polarization direction from the first pixel block. An image processing apparatus (20) acquires a captured image generated by the imaging apparatus (10), and a correction coefficient calculation unit (40) calculates a correction coefficient that absorbs a sensitivity difference between polarization pixels and non-polarization pixels, on the basis of an image of the second pixel block. A polarization information generation unit (50) generates polarization information indicating the polarization characteristics of the object included in the captured image, using a non-polarization image and polarization component images generated by a demosaicing unit (30) and the correction coefficient calculated by the correction coefficient calculation unit (40). The correction coefficient of the sensitivity difference between the polarization pixels and the non-polarization pixels can be acquired without any calibration.

First claim

The invention claimed is:

1. An imaging apparatus, comprising:
an imaging unit that has a first pixel block and a second pixel block provided in a pixel region, wherein
the first pixel block including polarization pixels with two different polarization directions and a non-polarization pixel,
the second pixel block incuding includes polarization pixels with three different polarization directions or polarization pixels with polarization directions orthogonal to one another, and a non-polarization pixel, and
two polarization directions of the three different polarization directions and one polarization direction of the polarization directions orthogonal to one another are polarization directions equal to a polarization direction of a polarization pixel of the polarization pixels in the first pixel block.

2. The imaging apparatus according to claim 1 , wherein a number of the second pixel blocks is smaller than a number of the first pixel blocks.

3. The imaging apparatus according to claim 2 , wherein the second pixel block is in a region different from a polarization information generation target region.

4. The imaging apparatus according to claim 2 , wherein a plurality of the second pixel blocks is in a radial direction from a position corresponding to an optical axis position of an imaging lens.

5. The imaging apparatus according to claim 2 , wherein a plurality of the second pixel blocks is at specific intervals.

6. An image processing apparatus, comprising:
central processing unit (CPU) configured to calculate a correction coefficient of a sensitivity difference between a non-polarization pixel and a polarization pixel, based on polarization pixels with at least two different polarization directions and non-polarization pixels in a captured image that includes the polarization pixels with a plurality of polarization directions and the non-polarization pixels.

7. The image processing apparatus according to claim 6 , wherein
the captured image is a captured image generated by an imaging unit that has a first pixel block and a second pixel block provided in a pixel region,
the first pixel block includes polarization pixels with two different polarization directions and a non-polarization pixel,
the second pixel block polarization pixels with a plurality of polarization directions including a different polarization direction from the first pixel block.

8. The image processing apparatus according to claim 7 , wherein
the second pixel block includes polarization pixels with three different polarization directions, and
the CPU is further configured to calculate the correction coefficient based on a basis of pixel values of the non-polarization pixel and the polarization pixels with the three different polarization directions.

9. The image processing apparatus according to claim 7 , wherein
the second pixel block includes polarization pixels with polarization directions orthogonal to one another, and
the CPU is further configured to calculate the correction coefficient based on pixel values of the non-polarization pixel and the polarization pixels with the polarization directions orthogonal to one another.

10. The image processing apparatus according to claim 6 , wherein CPU is further configured to calculate the correction coefficient again at a specific timing.

11. The image processing apparatus according to claim 10 , wherein the specific timing is timing at which imaging lenses are replaced in an imaging apparatus that generates the captured image.

12. An image processing apparatus according to claim 6 , wherein:
the CPU is further configured to:
generate a non-polarization image and a polarization component image for each polarization direction from the captured image; and
generate polarization information that indicates polarization characteristics of an object included in the captured image, based on the polarization component image and the non-polarization image, and the calculated correction coefficient.

13. An image processing method, comprising:
calculating a correction coefficient of a sensitivity difference between a non-polarization pixel and a polarization pixel, based on polarization pixels with at least two different polarization directions and non-polarization pixels in a captured image including the polarization pixels with a plurality of polarization directions and the non-polarization pixels.

Assignees

• Sony Group Corp

Inventors

• Hirasawa Yasutaka
• Kondo Yuhi
• Sun Legong
• Ono Taishi

Classifications

• H04N23/951 Primary

  by using two or more images to influence resolution, frame rate or aspect ratio · CPC title

• G06T1/0007

  Image acquisition · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T2207/10024

  Color image · CPC title

• H04N17/002 Primary

  for television cameras · CPC title