Image processing device, image processing method, image capturing apparatus, and image processing system

Patent metadata

Field	Value
Publication number	US-12283032-B2
Application number	US-202117794632-A
Country	US
Kind code	B2
Filing date	Feb 1, 2021
Priority date	Mar 10, 2020
Publication date	Apr 22, 2025
Grant date	Apr 22, 2025

Abstract

An image processing device includes: a determination means for determining an image capturing means used for capturing at least one particular portion in a combined image in which a first partial image captured using a first image capturing means and a second partial image captured using a second image capturing means are stitched together; and a correction means for applying image correction to the combined image based on a determination result of the determination means.

First claim

The invention claimed is:

1. An image processing device comprising:
determination circuitry configured to determine image capturing circuitry used for capturing at least one particular portion in a combined image in which a first partial image captured using first image capturing circuitry and a second partial image captured using second image capturing circuitry are stitched together; and
correction circuitry configured to
generate a correction table which includes for each set of coordinates in the combined image after an image correction, coordinates in the combined image before the image correction, a format designation value for designating a format of the correction, and an image correction amount of the image correction in the format designated using the format designation value; and
apply the image correction to the combined image based on the correction table to correct the combined image.

2. The image processing device according to claim 1 , wherein the image correction includes coordinate correction on a pixel constituting the combined image.

3. The image processing device according to claim 2 , further comprising:
first coordinate conversion circuitry for applying first coordinate conversion on coordinates of the at least one particular portion in the combined image before application of the coordinate correction; and
second coordinate conversion circuitry for applying second coordinate conversion on coordinates obtained by application of the coordinate correction, the second coordinate conversion being configured to apply coordinate conversion reversal of the first coordinate conversion.

4. The image processing device according to claim 3 , wherein the first coordinate conversion and the second coordinate conversion each include rotation conversion.

5. The image processing device according to claim 4 , wherein determination of the determination circuitry is performed based on coordinates obtained by application of the rotation conversion of the first coordinate conversion.

6. The image processing device according to claim 3 , wherein the combined image is an image in which distortion correction is applied to the first partial image and the second partial image and the first partial image and the second partial image are stitched together at a stitching position per area, and the first coordinate conversion and the second coordinate conversion each include stitching conversion based on the stitching position per area.

7. The image processing device according to claim 2 , wherein
the coordinate correction is performed based on a pixel position in the image capturing circuitry associated based on a determination result of the determination circuitry, and
information in which the particular portion in the combined image is associated with a pixel position in the first partial image or a pixel position in the second partial image is provided together with the combined image.

8. The image processing device according to claim 7 , wherein the coordinate correction is blur correction and is performed according to a correction amount corresponding to a difference in time of exposure to light within a frame, based on the image capturing circuitry determined by the determination circuitry and the pixel position.

9. The image processing device according to claim 1 , wherein the image correction includes pixel-value correction on a pixel constituting the combined image.

10. The image processing device according to claim 9 , wherein the pixel-value correction is performed based on coordinates of the particular portion in the combined image, or a pixel position in the first partial image or a pixel position in the second partial image associated with the particular portion in the combined image.

11. The image processing device according to claim 10 , wherein the pixel-value correction is light-intensity correction and is performed according to a correction amount corresponding to an image height, based on the image capturing circuitry determined by the determination circuitry, and coordinates of the particular portion in the combined image or the pixel position.

12. The image processing device according to claim 1 , further comprising:
an information reader to read information relating to the first image capturing circuitry and the second image capturing circuitry, wherein
the information relating to the first image capturing circuitry and the second image capturing circuitry includes at least one piece of information selected from information in which the particular portion in the combined image is associated with a pixel position in the first partial image or in the second partial image, information on coordinate conversion applied to the combined image, and information on variations in posture of the first image capturing circuitry and the second image capturing circuitry.

13. The image processing device according to claim 1 , further comprising:
output circuitry configured to output correction data that holds, for each of sets of coordinates in the combined image after correction, coordinates in the combined image before the correction, a format designation value for designating a format of the correction, and a correction amount of the correction in the format designated using the format designation value.

14. The image processing device according to claim 1 , wherein the image correction includes first image correction and second image correction, and the first image correction and the second image correction are directly applied in a superimposed manner, or are executed such that the first image correction is applied to obtain an intermediate image and the second image correction is applied to the intermediate image.

15. An image capturing apparatus, comprising:
the image processing device according to claim 1 ;
first image capturing circuitry for capturing a first partial image;
second image capturing circuitry for capturing a second partial image; and
recording circuitry for recording the combined image in which the first partial image and the second partial image are stitched together.

16. An image processing method for processing a combined image in which a first partial image captured using first image capturing circuitry and a second partial image captured using second image capturing circuitry are stitched together, the image processing method comprising:
determining image capturing circuitry used for capturing at least one particular portion in the combined image;
generating a correction table which includes, for each set of coordinates in the combined image after an image correction, coordinates in the combined image before the image correction, a format designation value for designating a format of the correction, and an image correction amount of the image correction in the format designated using the format designation value; and
applying the image correction on the combined image based on the correction table to correct the combined image.

17. The image processing method according to claim 16 , wherein the image correction includes one or both of coordinate correction on a pixel constituting the combined image and pixel-value correction on a pixel constituting the combined image.

18. A non-transitory computer readable medium storing computer readable code which, when executed by a computer system, cause the computer system to:
determine image capturing circuitry used for capturing at least one particular portion in a combined image in which a first partial image captured using first image capturing circuitry and a second part

Assignees

• Yoshida Kazuhiro
• Ricoh Co Ltd

Inventors

• Yoshida Kazuhiro

Classifications

• G06T5/73

  Deblurring; Sharpening · CPC title

• G06T2207/20212

  Image combination · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T3/12

  Panospheric to cylindrical image transformations · CPC title