Image processing device that synthesizes a plurality of images, method of controlling the same, storage medium, and image pickup apparatus

Patent metadata

Field	Value
Publication number	US-9648238-B2
Application number	US-201514867352-A
Country	US
Kind code	B2
Filing date	Sep 28, 2015
Priority date	Oct 6, 2014
Publication date	May 9, 2017
Grant date	May 9, 2017

Abstract

An image processing device that is capable of synthesizing a plurality of images by accurately correcting an image shift even in a photographing scene with a lot of external disturbances, such as noise. An image processing device synthesizes a plurality of images obtained by photographing an object that moves at a predetermined speed to thereby obtain a synthesized image. Each of the plurality of images is divided into a plurality of blocks, and a motion vector is detected from each of the blocks. Selection from the motion vectors is performed based on a focal length and an exposure interval applied when the plurality of images are obtained. Synthesis processing is performed by aligning the plurality of images based on the selected motion vectors, whereby the synthesized image is obtained.

First claim

What is claimed is:

1. An image processing device that synthesizes a plurality of images obtained by photographing an object that moves at a predetermined speed to thereby obtain a synthesized image, comprising:
a detection unit configured to divide each of the plurality of images into a plurality of blocks, and detect a motion vector from each of the blocks;
a selection unit configured to estimate an object maximum movement amount indicative of a maximum amount of a movement amount of the object between the plurality of images based on a focal length and an exposure interval which are applied when each of the plurality of images is obtained, and to select motion vectors having a scalar amount smaller than the object maximum movement amount; and
a synthesis unit configured to perform synthesis processing by aligning the plurality of images based on motion vectors selected by said selection unit to thereby obtain the synthesized image.

2. The image processing device according to claim 1 , wherein said detection unit selects a block in which a total sum of signal levels of edges extracted from each of the blocks is larger than a first threshold value set in advance, as a feature block, and detects the motion vector with respect to the feature block.

3. The image processing device according to claim 2 , wherein when the number of feature blocks is larger than a second threshold value set in advance, said detection unit detects the motion vector with respect to each feature block.

4. The image processing device according to claim 1 , wherein said selection unit calculates a degree of variation in a scalar amount in all of the motion vectors, and selects the motion vectors based on the degree of variation.

5. The image processing device according to claim 4 , wherein the degree of variation is an average value and a standard deviation of the scalar amount.

6. The image processing device according to claim 5 , wherein said selection unit sets a selection range for selecting the motion vectors based on the average value and the standard deviation, and selects a motion vector having a scalar amount within the selection range.

7. The image processing device according to claim 1 , wherein said selection unit calculates a degree of variation in the scalar amount in all of the motion vectors selected based on the object maximum movement amount, and further selects motion vectors, from the motion vectors selected based on the object maximum movement amount, based on the calculated degree of variation.

8. The image processing device according to claim 7 , wherein the degree of variation is an average value and a standard deviation of the scalar amount.

9. The image processing device according to claim 8 , wherein said selection unit sets a selection range for selecting the motion vectors based on the average value and the standard deviation, and selects a motion vector having a scalar amount within the selection range.

10. The image processing device according to claim 1 , wherein the object is a star in the sky.

11. An image pickup apparatus comprising:
an image pickup unit configured to photograph an object that moves at a predetermined speed to thereby obtain a plurality of images;
an image processing device configured to synthesize the plurality of images to thereby obtain a synthesized image; and
a recording unit configured to record the synthesized image obtained by said image processing device in a memory,
wherein said image processing device includes:
a detection unit configured to divide each of the plurality of images into a plurality of blocks, and detect a motion vector from each of the blocks;
a selection unit configured to estimate an object maximum movement amount indicative of a maximum amount of a movement amount of the object between the plurality of images based on a focal length and an exposure interval which are applied when each of the plurality of images is obtained, and to select a motion vector having a scalar amount smaller than the object maximum movement amount; and
a synthesis unit configured to perform synthesis processing by aligning the plurality of images based on motion vectors selected by said selection unit to thereby obtain the synthesized image.

12. A method of controlling an image processing device that synthesizes a plurality of images obtained by photographing an object that moves at a predetermined speed to thereby obtain a synthesized image, comprising:
dividing each of the plurality of images into a plurality of blocks, and detecting a motion vector from each of the blocks;
estimating an object maximum movement amount indicative of a maximum amount of a movement amount of the object between the plurality of images based on a focal length and an exposure interval which are applied when each of the plurality of images is obtained;
selecting a motion vector having a scalar amount smaller than the object maximum movement amount; and
performing synthesis processing by aligning the plurality of images based on motion vectors selected by said selecting to thereby obtain the synthesized image.

13. A non-transitory computer-readable storage medium storing a computer-executable program for executing a method of controlling an image processing device that synthesizes a plurality of images obtained by photographing an object that moves at a predetermined speed to thereby obtain a synthesized image, wherein the method comprises:
dividing each of the plurality of images into a plurality of blocks, and detecting a motion vector from each of the blocks;
estimating an object maximum movement amount indicative of a maximum amount of a movement amount of the object between the plurality of images based on a focal length and an exposure interval which are applied when each of the plurality of images is obtained;
selecting a motion vector having a scalar amount smaller than the object maximum movement amount; and
performing synthesis processing by aligning the plurality of images based on motion vectors selected by said selecting to thereby obtain the synthesized image.

Assignees

• Canon Kk

Inventors

• Takahashi Yoichi

Classifications

• G06T7/223

  using block-matching · CPC title

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title

• H04N23/683

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N23/6811 Primary

  based on the image signal · CPC title

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title