Imaging device, imaging method, program, and image processing device

Patent metadata

Field	Value
Publication number	US-2018115752-A1
Application number	US-201615567318-A
Country	US
Kind code	A1
Filing date	Apr 25, 2016
Priority date	May 7, 2015
Publication date	Apr 26, 2018
Grant date	—

Abstract

The present disclosure related to an imaging device, an imaging method, a program, and an image processing device that are capable of generating a color image having high color reproducibility. The imaging device includes: an imaging element configured to read out, in a first readout period that is part of a first subframe period in a single frame period, first electric charge signals of pixels with a focal-plane readout method, read out, in a second readout period that is part of a second subframe period in the single frame period, second electric charge signals of the pixels with the focal-plane readout method, generate a first subframe image on the basis of the first electric charge signals, and generate a second subframe image on the basis of the second electric charge signals; an irradiation control unit configured to perform control such that, in an irradiation period that is the whole or part of a period other than the first readout period in the first subframe period, infrared light is radiated and, in a period other than the irradiation period, infrared light is not radiated; and a color image generation unit configured to generate a color image at the predetermined frame rate on the basis of the first subframe image and the second subframe image. For example, the present disclosure can be applied to a monitoring camera.

First claim

1 . An imaging device comprising:
an imaging element configured to read out, in a first readout period that is part of a first subframe period in a single frame period corresponding to a predetermined frame rate, first electric charge signals of pixels in each line with a focal-plane readout method, read out, in a second readout period that is part of a second subframe period in the single frame period, second electric charge signals of the pixels in each line with the focal-plane readout method, generate a first subframe image on the basis of the first electric charge signals, and generate a second subframe image on the basis of the second electric charge signals; an irradiation control unit configured to perform control such that, in an irradiation period that is the whole or part of a period other than the first readout period in the first subframe period, an imaging range is irradiated with infrared light and, in a period other than the irradiation period, the imaging range is not irradiated with infrared light; and a color image generation unit configured to generate a color image at the predetermined frame rate on the basis of the first subframe image and the second subframe image.

2 . The imaging device according to claim 1 ,
wherein the first subframe period is shorter than the second subframe period.

3 . The imaging device according to claim 1 ,
wherein pixels in all lines are simultaneously exposed in the period other than the first readout period in the first subframe period and a period other than the second readout period in the second subframe period.

4 . The imaging device according to claim 1 ,
wherein the color image generation unit further includes
a first image conversion unit configured to extract luminance information from the first subframe image,
a second image conversion unit configured to extract color difference information from the second subframe image, and
a third image conversion unit configured to convert the luminance information of the first subframe image and the color difference information of the second subframe image into color information of the color image.

5 . The imaging device according to claim 4 , wherein
the second image conversion unit also extracts luminance information from the second subframe image, and the third image conversion unit converts the luminance information of the first subframe image, the luminance information being corrected by using the luminance information of the second subframe image, and the color difference information of the second subframe image into the color information of the color image.

6 . The imaging device according to claim 1 , wherein
the color image generation unit further includes
a first 3DNR unit configured to perform 3DNR processing including time-direction processing with respect to the first subframe image, and
a second 3DNR unit configured to perform 3DNR processing including time-direction processing with respect to the second subframe image, and
a feedback factor in the time-direction processing in the first 3DNR unit is different from a feedback factor in the time-direction processing in the second 3DNR unit.

7 . The imaging device according to claim 6 ,
wherein the second 3DNR unit performs the 3DNR processing with respect to the second subframe image by using a moving-subject detection result supplied from the first 3DNR unit.

8 . The imaging device according to claim 7 ,
wherein the color image generation unit further includes
a 2DNR unit configured to perform 2DNR processing with respect to the second subframe image by using an edge detection result supplied from the first 3DNR unit.

9 . The imaging device according to claim 1 ,
wherein a light-receiving surface of the imaging element is covered with a color filter in a Bayer array, an RGB-IR filter, or an RGB-W filter.

10 . The imaging device according to claim 1 , further comprising
an infrared light irradiation unit configured to perform irradiation with the infrared light.

11 . An imaging method comprising:
a generation step of reading out, in a first readout period that is part of a first subframe period in a single frame period corresponding to a predetermined frame rate, first electric charge signals of pixels in each line with a focal-plane readout method, reading out, in a second readout period that is part of a second subframe period in the single frame period, second electric charge signals of the pixels in each line with the focal-plane readout method, generating a first subframe image on the basis of the first electric charge signals, and generating a second subframe image on the basis of the second electric charge signals; an irradiation control step of performing control such that, in an irradiation period that is the whole or part of a period other than the first readout period in the first subframe period, an imaging range is irradiated with infrared light and, in a period other than the irradiation period, the imaging range is not irradiated with infrared light; and a color image generation step of generating a color image at the predetermined frame rate on the basis of the first subframe image and the second subframe image.

12 . A program for causing a computer to execute processing comprising
a generation step of reading out, in a first readout period that is part of a first subframe period in a single frame period corresponding to a predetermined frame rate, first electric charge signals of pixels in each line with a focal-plane readout method, reading out, in a second readout period that is part of a second subframe period in the single frame period, second electric charge signals of the pixels in each line with the focal-plane readout method, generating a first subframe image on the basis of the first electric charge signals, and generating a second subframe image on the basis of the second electric charge signals, an irradiation control step of performing control such that, in an irradiation period that is the whole or part of a period other than the first readout period in the first subframe period, an imaging range is irradiated with infrared light and, in a period other than the irradiation period, the imaging range is not irradiated with infrared light, and a color image generation step of generating a color image at the predetermined frame rate on the basis of the first subframe image and the second subframe image.

13 . An image processing device that accepts input of a first subframe image and a second subframe image from an imaging element configured to read out, in a first readout period that is part of a first subframe period in a single frame period corresponding to a predetermined frame rate, first electric charge signals of pixels in each line with a focal-plane readout method, read out, in a second readout period that is part of a second subframe period in the single frame period, second electric charge signals of the pixels in each line with the focal-plane readout method, generate the first subframe image on the basis of the first electric charge signals, and generate the second subframe image on the basis of the second electric charge signals, the image processing device comprising:
an irradiation control unit configured to perform control such that, in an irradiation period that is the whole or part of a period other than the first readout period in the first subframe period, an imaging range is irradiated with infrared light and, in a period other than the irradiation period, the imaging range is not irradiated with infrared light; and a color image generation unit configured to generate a color image at the predetermined fram

Assignees

• Sony Semiconductor Solutions Corp

Inventors

• Sato Yasushi

Classifications

• H04N25/133

  including elements passing panchromatic light, e.g. filters passing white light · CPC title

• H04N25/131

  including elements passing infrared wavelengths · CPC title

• H04N25/135

  based on four or more different wavelength filter elements · CPC title

• H04N23/84 Primary

  for processing colour signals · CPC title

• H04N25/531

  by controlling rolling shutters in CMOS SSIS · CPC title