Image pickup apparatus that compensates for flash band, control method therefor, storage medium, and video processing apparatus

Patent metadata

Field	Value
Publication number	US-10187581-B2
Application number	US-201715615993-A
Country	US
Kind code	B2
Filing date	Jun 7, 2017
Priority date	Jun 15, 2016
Publication date	Jan 22, 2019
Grant date	Jan 22, 2019

Abstract

An image pickup apparatus which satisfactorily compensates for a flash band appearing due to an external flash and prevents a row insensitive to the flash from appearing in a corrected image. The flash band appearing in a plurality of frames consecutive in terms of time is detected based on an image signal output from an image pickup device which sequentially starts exposure and sequentially reads out signals for each row of pixels. An image in at least one of those frames is corrected to a full-screen flash image. When a width of the flash band is smaller than the number of rows in one frame, a row in which the flash band does not appear is interpolated using a row in which the flash band appears and which immediately precedes or immediately succeeds the row in which the flash band does not appear.

First claim

What is claimed is:

1. An image pickup apparatus comprising:
an image pickup device configured to sequentially start exposure and sequentially read out signals for each row of pixels; and
at least one processor or circuit configured to function as the following units:
a detection unit configured to, based on an image signal output from the image pickup device, detect a flash band that appears in a plurality of frames consecutive in terms of time due to an external flash;
a correction unit configured to correct an image in at least one of the plurality of frames in which the flash band was detected by the detection unit to a full-screen flash image; and
a determination unit configured to determine whether a width of the flash band detected by the detection unit is equal to or greater than the number of rows in one frame,
wherein when the determination unit determines that the width of the flash band is not equal to or greater than the number of rows in one frame, the correction unit interpolates a row in which the flash band does not appear using a row in which the flash band appears and which immediately precedes or immediately succeeds the row in which the flash band does not appear.

2. The image pickup apparatus according to claim 1 , wherein according to whether the external flash is fired within a time period over which an electronic shutter of the image pickup device is on, the determination unit determines whether the width of the flash band is equal to or greater than the number of rows in one frame.

3. The image pickup apparatus according to claim 1 , wherein when a position at which an electronic shutter of the image pickup device was turned on is such a position that an exposure time period of the image pickup device is shorter than a value determined in advance, the correction unit does not interpolate the row in which the flash band does not appear.

4. The image pickup apparatus according to claim 1 , wherein when an average luminance value of the corrected image is greater than a value determined in advance, the correction unit does not interpolate the row in which the flash band does not appear.

5. The image pickup apparatus according to claim 4 , wherein the average luminance value of the corrected image is an average luminance value of the corrected image as a whole or an average luminance value in a row at a boundary between the row in which the flash band appears and the row in which the flash band does not appear.

6. The image pickup apparatus according to claim 1 , wherein the correction unit interpolates the row in which the flash band does not appear by copying a row in which the flash band appears and which immediately precedes or immediately succeeds the row in which the flash band does not appear.

7. The image pickup apparatus according to claim 1 , wherein the correction unit interpolates the row in which the flash band does not appear by averaging rows in which the flash band appears and which immediately precede and immediately succeed the row in which the flash band does not appear.

8. A video processing apparatus that corrects a video comprising a plurality of frames, comprising:
at least one processor or circuit configured to function as the following units:
a detection unit configured to detect a flash band that appears in a plurality of frames consecutive in terms of time among the plurality of frames;
a correction unit configured to correct an image in at least one of the plurality of frames in which the flash band was detected to a flash image by combining regions where the flash band appears in respective ones of the plurality of frames in which the flash band was detected; and
an output unit configured to output a video corrected by the correction unit,

wherein when a total number of rows in a region where the flash band appears in the plurality of frames in which the flash band was detected is smaller than the number of frames in one frame, the correction unit interpolates an image in a row at a boundary of the flash band using image data in the region where the flash band appears.

9. The video processing apparatus according to claim 8 , wherein the correction unit interpolates the row at the boundary of the flash band by copying a row in which the flash band appears and which immediately precedes or immediately succeeds the row at the boundary of the flash band.

10. The video processing apparatus according to claim 8 , wherein the correction unit interpolates the row at the boundary of the flash band by averaging rows in which the flash band appears and which immediately precede and immediately succeed the row at the boundary of the flash band.

11. A control method for an image pickup apparatus with an image pickup device that sequentially starts exposure and sequentially reads out signals for each row of pixels, comprising:
a detection step of, based on an image signal output from the image pickup device, detecting a flash band that appears in a plurality of frames consecutive in terms of time due to an external flash;
a correction step of correcting an image in at least one of the plurality of frames in which the flash band was detected in the detection step to a full-screen flash image; and
a determination step of determining whether a width of the flash band detected in the detection step is equal to or greater than the number of rows in one frame,
wherein when it is determined that the width of the flash band is not equal to or greater than the number of rows in one frame, a row in which the flash band does not appear is corrected using a row in which the flash band appears and which immediately precedes or immediately succeeds the row in which the flash band does not appear.

12. A non-transitory computer-readable storage medium storing a program for causing a computer to execute a control method for an image pickup apparatus with an image pickup device that sequentially starts exposure and sequentially reads out signals for each row of pixels, the control method comprising:
a detection step of, based on an image signal output from the image pickup device, detecting a flash band that appears in a plurality of frames consecutive in terms of time due to an external flash;
a correction step of correcting an image in at least one of the plurality of frames in which the flash band was detected in the detection step to a full-screen flash image; and
a determination step of determining whether a width of the flash band detected in the detection step is equal to or greater than the number of rows in one frame,
wherein when it is determined that the width of the flash band is not equal to or greater than the number of rows in one frame, a row in which the flash band does not appear is corrected using a row in which the flash band appears and which immediately precedes or immediately succeeds the row in which the flash band does not appear.

Assignees

• Canon Kk

Inventors

• Kobuse Takenori

Classifications

• H04N23/71

  Circuitry for evaluating the brightness variation · CPC title

• H04N25/531

  by controlling rolling shutters in CMOS SSIS · CPC title

• H04N25/61

  the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4" · CPC title

• H04N23/70 Primary

  Circuitry for compensating brightness variation in the scene · CPC title

• H04N25/46

  by combining or binning pixels · CPC title