Imaging device and image data generation method

Patent metadata

Field	Value
Publication number	US-10691002-B2
Application number	US-201816013942-A
Country	US
Kind code	B2
Filing date	Jun 21, 2018
Priority date	Jan 18, 2016
Publication date	Jun 23, 2020
Grant date	Jun 23, 2020

Abstract

Provided are an imaging device and an image data generation method which are capable of reducing noise generated in an image in which crosstalk is removed. In an imaging device (1) that captures images corresponding to optical systems at one time by using an imaging lens (10) including a plurality of optical system of which imaging characteristics are different and an image sensor (100) including a plurality of light receiving sensors of which crosstalk ratios are different in each pixel, the number (m) of light receiving sensors included in each pixel of the image sensor (100) is larger than the number (n) of optical systems included in the imaging lens (10) (m>n). Accordingly, it is possible to reduce noise generated in an image in which crosstalk is removed.

First claim

What is claimed is:

1. An imaging device comprising:
an imaging lens that includes n number of optical systems of which imaging characteristics are different in which n is an integer satisfying n>1;
an image sensor that includes m number of light receiving sensors of which crosstalk ratios are different in each pixel in which m is an integer satisfying m>n; and
a processor configured to:
obtain image signals from the light receiving sensors of each pixel of the image sensor, and generates m number of primary image data items; and
generate n number of secondary image data items corresponding to the optical systems by performing crosstalk removal processing on the m number of primary image data items for each pixel.

2. The imaging device according to claim 1 ,
wherein, in a case where pixel values of the primary image data items are A1, A2, . . . , and Am, pixel values of the secondary image data items are B1, B2, . . . , and Bn, and the crosstalk ratios are C1=C11:C12: . . . :C1n, C2=C21:C22: . . . :C2n, . . . , and Cm=Cm1:Cm2: . . . :Cmn, the processor generates the secondary image data items by solving the following simultaneous equations A1=C11*B1+C12*B2+ . . . +C1n*Bn, A2=C21*B1+C22*B2+ . . . +C2n*Bn, . . . , and Am=Cm1*B1+Cm2*B2+ . . . +Cmn*Bn, as the crosstalk removal processing.

3. The imaging device according to claim 1 ,
wherein, in a case where a matrix of m rows×1 column in which pixel values A1, A2, . . . , and Am of the primary image data items are elements is A, a matrix of n rows×1 column in which pixel values B1, B2, . . . , and Bn of the secondary image data items are elements is B, a matrix of m rows×n columns in which the crosstalk ratios C1=C11:C12: . . . :C1n, C2=C21:C22: . . . :C2n, . . . , and Cm=Cm1:Cm2: . . . :Cmn are elements is C, and an inverse matrix of the C is C −1 , the processor generates the secondary image data items by solving the following matrix equation B=C −1 *A, as the crosstalk removal processing.

4. The imaging device according to claim 3 , wherein the processor further solves the matrix equation by using the information of the inverse matrix C −1 stored therein.

5. The imaging device according to claim 4 ,
wherein the processor stores the information of the inverse matrix C −1 for each pixel.

6. The imaging device according to claim 1 ,
wherein the processor further obtains information of an imaging scene,
wherein the processor changes the number of primary image data items to be used in the generation of the secondary image data items depending on the imaging scene.

7. The imaging device according to claim 2 ,
wherein the processor further obtains information of an imaging scene,
wherein the processor changes the number of primary image data items to be used in the generation of the secondary image data items depending on the imaging scene.

8. The imaging device according to claim 3 ,
wherein the processor further obtains information of an imaging scene,
wherein the processor changes the number of primary image data items to be used in the generation of the secondary image data items depending on the imaging scene.

9. The imaging device according to claim 4 ,
wherein the processor further obtains information of an imaging scene,
wherein the processor changes the number of primary image data items to be used in the generation of the secondary image data items depending on the imaging scene.

10. The imaging device according to claim 5 ,
wherein the processor further obtains information of an imaging scene,
wherein the processor changes the number of primary image data items to be used in the generation of the secondary image data items depending on the imaging scene.

11. The imaging device according to claim 6 ,
wherein the processor specifies the imaging scene by analyzing the primary image data items.

12. The imaging device according to claim 1 ,
wherein the light receiving sensor includes
a photoelectric conversion element,
a microlens that forms an image of an emission pupil of the imaging lens on the photoelectric conversion element, and
a light shielding mask that is disposed between the microlens and the light receiving sensor, and
a difference in shape of the light shielding mask and/or image forming characteristics of the microlens causes a difference in the crosstalk ratios.

13. The imaging device according to claim 1 ,
wherein the light receiving sensor includes
a photoelectric conversion element, and
a microlens that forms an image of an emission pupil of the imaging lens on the photoelectric conversion element, and
a difference in image forming characteristics of the microlens causes a difference in the crosstalk ratios.

14. The imaging device according to claim 1 ,
wherein the n number of optical systems included in the imaging lens have different focal lengths.

15. The imaging device according to claim 1 ,
wherein the n number of optical systems included in the imaging lens have different focusing distances.

16. The imaging device according to claim 1 ,
wherein the n number of optical systems included in the imaging lens have different transmission wavelength characteristics.

17. The imaging device according to claim 14 ,
wherein the n number of optical systems included in the imaging lens are concentrically disposed.

18. An image data generation method using an imaging device having an imaging lens and an image sensor,
wherein the imaging lens that includes n number of optical systems of which the imaging characteristics are different in which n is an integer satisfying n>1, the method comprising:
exposing the image sensor, wherein the image sensor includes m number of light receiving sensors of which crosstalk ratios are different in each pixel in which m is an integer satisfying m>n;
obtaining image signals from the light receiving sensors of each pixel of the image sensor and generating m number of primary image data items; and
generating n number of secondary image data items corresponding to the optical systems by performing crosstalk removal processing on the m number of primary image data items for each pixel.

19. The image data generation method according to claim 18 ,
wherein, in a case where pixel values of the primary image data items are A1, A2, . . . , and Am, pixel values of the secondary image data items are B1, B2, . . . , and Bn, and the crosstalk ratios are C1=C11:C12: . . . :C1n, C2=C21:C22: . . . :C2n, . . . , and Cm=Cm1:Cm2: . . . :Cmn, the secondary image data items are generated by solving the following simultaneous equations A1=C11*B1+C12*B2+ . . . +C1n*Bn, A2=C21*B1+C22*B2+ . . . +C2n*Bn, . . . , and Am=Cm1*B1+Cm2*B2+ . . . +Cmn*Bn, as the crosstalk removal processing.

20. The image data generation method according to claim 18 ,
wherein, in a case where a matrix of m rows×1 column in which pixel values A1, A2, . . . , and Am of the primary image data items are elements is A, a matrix of n rows×1 column in which pixel values B1, B2, . . . , and Bn of the secondary image data items are elements is B, a matrix of m rows×n columns in which the crosstalk ratios C1=C11:C12: . . . :C1n, C2=C21:C22: . . . :C2n, . . . , and Cm=Cm1:Cm2: . . . :Cmn are elements is C, and an inverse matrix of the C is C −1 , the secondary image data items are generated by solving the following matrix equation B=C −1 *A, as the crosstalk removal processing.

Assignees

• Fujifilm Corp

Inventors

• Ono Shuji

Classifications

• H04N25/62

  Detection or reduction of noise due to excess charges produced by the exposure, e.g. smear, blooming, ghost image, crosstalk or leakage between pixels · CPC title

• H04N23/55

  Optical parts specially adapted for electronic image sensors; Mounting thereof · CPC title

• H04N23/957

  Light-field or plenoptic cameras or camera modules · CPC title

• H10F39/8063

  Microlenses · CPC title

• H10F39/8057

  Optical shielding · CPC title