Image processing apparatus, system, image processing method, and non-transitory recording medium

Patent metadata

Field	Value
Publication number	US-10521891-B2
Application number	US-201815908946-A
Country	US
Kind code	B2
Filing date	Mar 1, 2018
Priority date	Mar 14, 2017
Publication date	Dec 31, 2019
Grant date	Dec 31, 2019

Abstract

An apparatus, system, and method of image processing, each of which: acquires a plurality of images respectively captured with a plurality of imaging elements, the plurality of images respectively corresponding to a plurality of frames of a video; calculates, for a current frame, an evaluation value to be used for evaluating each of the plurality of images using a pixel value of at least one pixel in an overlapping area of the plurality of images; determines whether an evaluation value of each of the plurality of images in a frame preceding the current frame is stored in a memory; determines at least one correction target image from the plurality of images based on the evaluation value calculated for the current frame and a determination result indicating whether the evaluation value in the preceding frame is stored; calculates a plurality of correction amounts to be used for correcting a plurality of evaluation values in the overlapping area of the correction target image, based on the evaluation value of the correction target image; and corrects the correction target image based on the plurality of correction amounts.

First claim

The invention claimed is:

1. An image processing apparatus comprising:
processing circuitry configured to,
acquire a plurality of images respectively captured with a plurality of imaging elements, the plurality of images respectively corresponding to a plurality of frames of a video,
calculate, for a current frame, an evaluation value to be used for evaluating each of the plurality of images using a pixel value of at least one pixel in an overlapping area of the plurality of images,
determine at least one correction target image from the plurality of images based on the evaluation value calculated for the current frame and an evaluation value in a preceding frame stored in a memory such that the correction target image is selected from among the plurality of images associated with the current frame by, for each of the plurality of images associated with the current frame, evaluating a difference between (i) the evaluation value calculated for a respective one of the plurality of images associated with the current frame and (ii) the evaluation value in the preceding frame,
calculate a plurality of correction amounts to be used for correcting a plurality of evaluation values in the overlapping area of the correction target image, based on the evaluation value of the correction target image, and
correct the correction target image based on the plurality of correction amounts.

2. The image processing apparatus of claim 1 ,
wherein the processing circuitry is configured to determine the correction target image for the current frame by referring to the evaluation value of at least one of the plurality of images in the preceding frame.

3. The image processing apparatus of claim 2 ,
wherein the at least one of the plurality of images in the preceding frame having the evaluation value that is referred is an image that has not been corrected.

4. The image processing apparatus of claim 2 ,
wherein the processing circuitry is configured to perform a time-axis correction process of correcting the plurality of correction amounts between adjacent ones of the plurality of frames captured sequentially with a same one of the plurality of imaging elements.

5. The image processing apparatus of claim 4 , wherein the processing circuitry is configured to perform the time-axis correction process of correcting the plurality of correction amounts between the adjacent ones of the plurality of frames captured sequentially with the same one of the plurality of imaging elements such that the processing circuitry reduces variations of the plurality of correction amounts between frames by performing inter-frame correction on a time-axis.

6. The image processing apparatus of claim 1 , wherein the overlapping area is set in pixel units.

7. The image processing apparatus of claim 1 ,
wherein the processing circuitry is further configured to combine the plurality of images including the correction target image that has been corrected to generate a spherical image.

8. A system comprising:
the image processing apparatus of claim 1 ; and
the plurality of imaging elements configured to respectively capture the plurality of images.

9. The image processing apparatus of claim 1 , wherein the plurality of correction amounts indicate changes in one or more of luminance and color.

10. An image processing method comprising:
acquiring a plurality of images respectively captured with a plurality of imaging elements, the plurality of images respectively corresponding to a plurality of frames of a video;
calculating, for a current frame, an evaluation value to be used for evaluating each of the plurality of images using a pixel value of at least one pixel in an overlapping area of the plurality of images;
determining at least one correction target image from the plurality of images based on the evaluation value calculated for the current frame and an evaluation value in a preceding frame stored in a memory such that the correction target image is selected from among the plurality of images associated with the current frame by, for each of the plurality of images associated with the current frame, evaluating a difference between (i) the evaluation value calculated for a respective one of the plurality of images associated with the current frame and (ii) the evaluation value in the preceding frame;
calculating a plurality of correction amounts to be used for correcting a plurality of evaluation values in the overlapping area of the correction target image, based on the evaluation value of the correction target image; and
correcting the correction target image based on the plurality of correction amounts.

11. The image processing method of claim 10 , wherein the determining includes:
referring to the evaluation value of at least one of the plurality of images in the preceding frame to determine the correction target image in the current frame, when the determination result indicates that the evaluation value in the preceding frame is stored.

12. The image processing method of claim 11 ,
wherein the at least one of the plurality of images in the preceding frame having the evaluation value that is referred is an image that has not been corrected.

13. The image processing method of claim 10 , further comprising:
performing a time-axis correction process of correcting the plurality of correction amounts between adjacent ones of the plurality of frames captured sequentially with a same one of the plurality of imaging elements.

14. The image processing method of claim 13 , wherein the performing the time-axis correction process performs the time-axis correction process of correcting the plurality of correction amounts between the adjacent ones of the plurality of frames captured sequentially with the same one of the plurality of imaging elements such that the processing circuitry reduces variations of the plurality of correction amounts between frames by performing inter-frame correction on a time-axis.

15. The image processing method of claim 10 , wherein the plurality of correction amounts indicate changes in one or more of luminance and color.

16. A non-transitory recording medium storing a program for causing one or more processors to execute an image processing method, the image processing method comprising:
acquiring a plurality of images respectively captured with a plurality of imaging elements, the plurality of images respectively corresponding to a plurality of frames of a video;
calculating, for a current frame, an evaluation value to be used for evaluating each of the plurality of images using a pixel value of at least one pixel in an overlapping area of the plurality of images;
determining at least one correction target image from the plurality of images based on the evaluation value calculated for the current frame an evaluation value in a preceding frame stored in a memory such that the correction target image is selected from among the plurality of images associated with the current frame by, for each of the plurality of images associated with the current frame, evaluating a difference between (i) the evaluation value calculated for a respective one of the plurality of images associated with the current frame and (ii) the evaluation value in the preceding frame;
calculating a plurality of correction amounts to be used for correcting a plurality of evaluation values in the overlapping area of the correction target image, based on the evaluation value of the correction target image; and
correcting the correction target image based on the plurality of correction amounts.

17. The non-transitor

Assignees

• Matsushita Kazufumi
• Ricoh Co Ltd

Inventors

• Matsushita Kazufumi

Classifications

• H04N23/45

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N23/698 Primary

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N25/61

  the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4" · CPC title

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title