Image stabilizing method and apparatus

Patent metadata

Field	Value
Publication number	US-9466095-B2
Application number	US-201514601467-A
Country	US
Kind code	B2
Filing date	Jan 21, 2015
Priority date	Jan 21, 2014
Publication date	Oct 11, 2016
Grant date	Oct 11, 2016

Abstract

Provided are an image stabilizing method and apparatus, which are capable of performing robust operations in a moving environment. The image stabilizing apparatus includes: an image pre-processor configured to receive a current image and a reference image corresponding to the current image; a feature point selector configured to select at least one feature point corresponding to a background from a plurality of feature points, comprising the background and at least one moving object, in the reference image; a motion estimator configured to estimate a motion of a current image, based on the selected feature point in the reference image and at least one corresponding feature point, corresponding to the selected feature point in the reference image, in the current image; and an image corrector configured to generate a stabilized current image by converting the current image by using the estimated motion.

First claim

What is claimed is:

1. An image stabilizing apparatus comprising:
an image pre-processor configured to receive a current image and a reference image corresponding to the current image;
a feature point selector configured to select at least one feature point corresponding to a background from a plurality of feature points, comprising the background and at least one moving object, in the reference image;
a motion estimator configured to estimate a motion of a current image, based on the selected feature point in the reference image and at least one corresponding feature point, corresponding to the selected feature point in the reference image, in the current image; and
an image corrector configured to generate a stabilized current image by converting the current image by using the estimated motion.

2. The apparatus of claim 1 , further comprising a foreground generator configured to generate a foreground image of the current image from a difference image between the current image and the reference image, and synchronize the foreground image with a next image,
wherein the feature point selector is configured to select the feature point in the reference image by removing at least one feature point, corresponding to a moving object in the synchronized foreground image, from among the feature points in the reference image.

3. The apparatus of claim 2 , wherein the foreground generator is configured to synchronize the foreground image with the next image by delaying the foreground image by one frame.

4. The apparatus of claim 2 , wherein the image pre-processor is configured to convert the current image and the reference image into gray images, and resize the gray images, and
wherein the foreground generator is configured to generate the foreground image based on a difference image between the resized images.

5. The image stabilizing apparatus of claim 4 , wherein the foreground generation is configured to remove noise by applying a blob filter to the difference image.

6. The apparatus of claim 1 , further comprising:
a first feature point extractor configured to extract the feature point of the reference image; and
a second feature point extractor configured to track the selected feature point of the reference image in the current image and extract the corresponding feature point of the current image.

7. The apparatus of claim 6 , wherein the image preprocessor is configured to convert the current image and the reference image into gray images and resize the gray images before the extraction of the feature point of the reference image, and
wherein the feature point extractor is configured to extract the feature point of the reference image using the resized gray images.

8. The apparatus of claim 1 , further comprising a foreground generator configured to generate a foreground image of a next image from a difference image between the stabilized current image and a next reference image.

9. The apparatus of claim 8 , wherein the next reference image is the same as the reference image.

10. The apparatus of claim 8 , wherein the feature point selector is configured to remove at least one feature point, corresponding to at least one moving object in the foreground image of the next image, from among a plurality of feature points, comprising the at least one moving object and a background, in the next reference image, select at least one feature point corresponding to the background in the next reference image, and detect at least one corresponding point in the next image by tracking the selected feature point in the next reference image,
wherein the motion estimator is configured to estimate a motion of the next image based on the detected corresponding point in the next image, and
wherein the image corrector is configured to generate a stabilized next image by converting the next image by using the estimated motion.

11. An image stabilizing method comprising:
receiving a current image and a reference image corresponding to the current image;
selecting at least one feature point corresponding to a background from a plurality of feature points, comprising the background and at least one moving object, in the reference image;
estimating a motion of the current image based on the selected feature point in the reference image and at least one corresponding feature point, corresponding to the selected feature point in the reference image, in the current image; and
generating a stabilized current image by converting the current image by using the estimated motion.

12. The method of claim 11 , further comprising
generating a foreground image of the current image from a difference image between the current image and the reference image; and
synchronizing the foreground image with a next image;
wherein the selecting at least one feature point in the reference image is performed by removing at least one feature point, corresponding to at least one moving object in the synchronized foreground image, from among the feature points in the reference image.

13. The method of claim 12 , wherein the synchronizing the foreground image with the next image is performed by delaying the foreground image by one frame.

14. The method of claim 12 , wherein the generating the foreground image comprises:
converting the current image and the reference image into gray images and resizing the gray images; and
generating the foreground image based on a difference image between the resized images.

15. The method of claim 14 , wherein the generating the foreground image comprises removing noise by applying blob filtering to the difference image.

16. The method of claim 11 , further comprising:
extracting the feature point of the reference image; and
tracking the selected feature point of the reference image in the current image and extracting the corresponding feature point of the current image.

17. The method of claim 16 , further comprising, before extracting the feature point of the reference image:
converting the reference image and the current image into gray images; and
resizing the gray images,
wherein the feature point of the reference image is extracted using the resized gray images.

18. The method of claim 11 , further comprising generating a foreground image of a next image from a difference image between the stabilized current image and a next reference image.

19. The method of claim 18 , wherein the next reference image is the same as the reference image.

20. The method of claim 18 , further comprising:
removing at least one feature point corresponding to at least one moving object in the foreground image of the next image from among a plurality of feature points, comprising the at least one moving object and a background, in the next reference image;
selecting at least one feature point corresponding to the background in the next reference image;
detecting at least one corresponding point in the next image by tracking the selected feature point in the next reference image;
estimating a motion of the next image based on the detected corresponding point in the next image; and
generating a stabilized next image by converting the next image by using the estimated motion.

Assignees

• Hanwha Techwin Co Ltd

Inventors

• Ryu Yeongeol
• Park Sangji
• Lee Hwalsuk
• Yin Shimin

Classifications

• G06T2207/20016

  Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform · CPC title

• G06T2207/20201

  Motion blur correction · CPC title

• G06T2207/20182

  Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering · CPC title

• G06T2207/30232

  Surveillance · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title