Image processing apparatus, image processing method, and storage medium

Patent metadata

Field	Value
Publication number	US-10484601-B2
Application number	US-201615249058-A
Country	US
Kind code	B2
Filing date	Aug 26, 2016
Priority date	Aug 31, 2015
Publication date	Nov 19, 2019
Grant date	Nov 19, 2019

Abstract

An image processing apparatus according to the present disclosure includes an acquisition unit configured to acquire a position for identifying a region in which a plurality of images are combined in an overlap region, the plurality of images being obtained in a partially-overlapping manner, a detection unit configured to detect an edge in vicinity of the position for identifying the region in the plurality of images, and a combining unit configured to combine the plurality of images to generate an output image by blending pixel values of pixels constituting the plurality of images, in vicinity of the position for identifying the region, wherein the combining unit sets a parameter for controlling the region in which the plurality of images are combined, according to a correlation between the position for identifying the region and the edge.

First claim

What is claimed is:

1. An image processing apparatus for generating a panoramic image by combining a plurality of images in a partially-overlapping manner, comprising:
one or more processors; and
one or more memory storing instructions executable by the one or more processors, wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:
acquire a position of a boundary line for combining a first image and a second image partially overlapping the first image in an overlap region, the first image and the second image being two of the plurality of images;
detect an edge component of an object in vicinity of the position of the boundary line in either of the first image and second image;
set a blend width of a region, in the overlap region and including the boundary line, where pixel values of pixels constituting the first image and pixel values of pixels constituting the second image are blended, wherein the blend width is set based on the position of the boundary line and the edge component; and
combine the first image and the second image by blending the pixel values of the pixels constituting the first image and the pixel values of the pixels constituting the second image, in the region of the set blend width in the overlap region.

2. The image processing apparatus according to claim 1 , wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:
in a case in which the position of the boundary line and the edge component match each other, set the blend width of the region in which the first image and the second image are combined, to be smaller than in a case in which the position of the boundary line and the edge component do not match each other.

3. The image processing apparatus according to claim 1 , wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:
calculate, as a correlation, a matching degree between a position and a direction of the boundary line and a position and a direction of the edge component.

4. The image processing apparatus according to claim 1 , wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:
calculate an evaluation value for evaluating a matching degree between the position of the boundary line and the edge component, and set the blend width according to the evaluation value.

5. The image processing apparatus according to claim 1 , wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:
acquire a maximum blend width comprising an upper limit of the blend width, and
set the blend width to the maximum blend width or smaller.

6. The image processing apparatus according to claim 5 , wherein the maximum blend width is set based on an instruction issued by a user.

7. The image processing apparatus according to claim 1 , wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:
detect, as the edge component, a difference in characteristic between two partial regions obtained by division in a direction of the boundary line.

8. The image processing apparatus according to claim 7 , wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:
divide vicinity of the boundary line into blocks, and set, for each block, a blend width so as to narrow the blend width in a case in which a difference in characteristic between partial regions is large, and to widen the blend width in a case in which a difference in characteristic between partial regions is small.

9. The image processing apparatus according to claim 1 , wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:
detect, as the edge component, an amount representing a degree of change in pixel value.

10. The image processing apparatus according to claim 1 , wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:
detect a frequency characteristic as the edge component.

11. The image processing apparatus according to claim 1 , wherein the image includes distance information representing a distance from a shooting viewpoint for each region represented by pixels, and
wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:
detect an edge in distance information related to a position and a direction of the boundary line in the first image and the second image.

12. An image processing method for generating a panoramic image by combining a plurality of images in a partially-overlapping manner, comprising:
acquiring a position of a boundary line for combining a first image and a second image partially overlapping the first image in an overlap region, the first image and the second image being two of the plurality of images;
detecting an edge component of an object in vicinity of the position of the boundary line in either of the first image and second image;
setting a blend width of a region, in the overlap region and including the boundary line, where pixel values of pixels constituting the first image and pixel values of pixels constituting the second image are blended, wherein the blend width is set based on the position of the boundary line and the edge component; and
combining the first image and the second image by blending the pixel values of the pixels constituting the first image and the pixel values of the pixels constituting the second image, in the region of the set blend width in the overlap region.

13. A non-transitory computer-readable storage medium storing a program for causing a computer to execute an image processing method for generating a panoramic image by combining a plurality of images in a partially-overlapping manner, the image processing method comprising:
acquiring a position of a boundary line for combining a first image and a second image partially overlapping the first image in an overlap region, the first image and the second image being two of the plurality of images;
detecting an edge component of an object in vicinity of the position of the boundary line in either of the first image and second image;
setting a blend width of a region, in the overlap region and including the boundary line, where pixel values of pixels constituting the first image and pixel values of pixels constituting the second image are blended, wherein the blend width is set based on the position of the boundary line and the edge component; and
combining the first image and the second image by blending the pixel values of the pixels constituting the first image and the pixel values of the pixels constituting the second image, in the region of the set blend width in the overlap region.

14. The image processing apparatus according to claim 1 , wherein the instructions, when executed by the one or more processors, cause the image processing apparatus to:
acquire a boundary image indicating whether each pixel is a pixel positioned on the boundary line,
generate pre-blend image, which is the same size as the boundary image, by selecting any of the pixels in the first image and the second image, and generate an edge image indicating whether each pixel is an edge by detecting an edge in the pre-blend image as the feature amount, and
determine whether a block including the boundary line in the boundary image matches a block including the edge in the edge image, and set the blend width according to the result

Assignees

• Canon Kk

Inventors

• Kikuta Kyohei

Classifications

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/698 Primary

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N5/2624 Primary

  for obtaining an image which is composed of whole input images, e.g. splitscreen · CPC title

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• H04N1/00

  Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof · CPC title