Image stabilizing apparatus and method based on a predicted movement position

Patent metadata

Field	Value
Publication number	US-9854171-B2
Application number	US-201715434873-A
Country	US
Kind code	B2
Filing date	Feb 16, 2017
Priority date	Mar 23, 2015
Publication date	Dec 26, 2017
Grant date	Dec 26, 2017

Abstract

Provided are an image stabilizing apparatus and method thereof. The image stabilizing apparatus performs image stabilization by using both an image sensor and a motion sensor. Image distortion and movement are stably corrected by using both the position of a feature point, which is extracted by the image sensor and image processing, and the movement position of the feature point, which is predicted by the motion sensor.

First claim

What is claimed is:

1. A vibration correcting method in a moving camera, the vibration correcting method comprising:
extracting a feature point in an input image of the moving camera;
predicting a position, to which a feature point extracted in a first image is to move due to vibration of the moving camera, by using a motion sensor;
generating a motion vector between the feature point extracted in the first image, the predicted position to which the feature point extracted in the first image is to move, and a feature point extracted in a second image captured by the moving camera after occurrence of the vibration; and
correcting vibration of the second image based on the motion vector,
wherein the feature point extracted in the first image is detected based on a physical movement amount of the moving camera measured by using the motion sensor, and the feature point extracted in the second image is detected based on an image sensor of the moving camera.

2. The vibration correcting method of claim 1 , wherein the first image comprises a vibration-corrected image.

3. The vibration correcting method of claim 1 , wherein the first image comprises a frame captured immediately before the occurrence of the vibration of the moving camera, and the second image comprises a frame captured immediately after the occurrence of the vibration of the moving camera.

4. The vibration correcting method of claim 3 , wherein the first image comprises an image with vibration correction already reflected.

5. The vibration correcting method of claim 1 , wherein the feature point extracted in the second image comprises a corresponding feature point corresponding to the feature point extracted in the first image.

6. The vibration correcting method of claim 1 , wherein the motion vector comprises a local motion vector or a global motion vector.

7. The vibration correcting method of claim 1 , wherein the motion sensor comprises an inertial measurement unit (IMU) sensor or a gyro sensor.

8. The vibration correcting method of claim 7 , wherein the IMU sensor predicts a position to which the feature point extracted in the first image is to move physically due to the vibration of the moving camera.

9. The vibration correcting method of claim 1 , wherein the moving camera comprises a mobile surveillance camera.

10. The vibration correcting method of claim 1 , wherein the correcting of the vibration of the second image comprises determining, based on distribution of frequencies detected in the vibration of the second image, whether the vibration of the second image is caused by a hand shake of a user of the moving camera or by external impact applied to the moving camera.

11. The vibration correcting method of claim 1 , wherein when the vibration of the moving camera is caused by wind, the vibration is divided by a high-pass filter (HPF).

12. A vibration correcting method in a moving camera, the vibration correcting method comprising:
extracting a feature point in an input image of the moving camera;
predicting a position, to which a feature point extracted in a first image is to move due to vibration of the moving camera, by using a motion sensor;
generating a motion vector between the feature point extracted in the first image, the predicted position to which the feature point extracted in the first image is to move, and a feature point extracted in a second image captured by the moving camera after occurrence of the vibration; and
correcting vibration of the second image based on the motion vector,
wherein the correcting the vibration of the second image comprises determining that a position of the feature point extracted in the second image is accurate, when a distance between the predicted position to which the feature point extracted in the first image is to move and the feature point extracted in the second image captured by the moving camera after the occurrence of the vibration is within a predetermined range, and correcting the vibration of the second image by using position information of the feature point extracted in the second image.

13. The vibration correcting method of claim 1 , wherein the motion sensor comprises an inertial measurement unit (IMU) sensor, and is attached outside the moving camera.

14. The vibration correcting method of claim 1 , wherein the moving camera comprises a robot.

15. The vibration correcting method of claim 1 , wherein the vibration comprises at least one of movements including user's handshake, rotation, in-plane rotation, three-axis rotation, translation, tilting, panning, and any other movement of the moving camera.

16. The vibration correcting method of claim 1 , wherein the moving camera comprises a wearable camera.

17. A vibration correcting method in a moving camera, the vibration correcting method comprising:
extracting a feature point in an input image of the moving camera;
predicting a position, to which a feature point extracted in a first image is to move due to vibration of the moving camera, by using a motion sensor;
generating a motion vector between the predicted position to which the feature point extracted in the first image is to move and a feature point extracted in a second image captured by the moving camera after occurrence of the vibration; and
correcting vibration of the second image based on the motion vector,
wherein the feature point extracted in the first image is detected based on a physical movement amount of the moving camera measured by using the motion sensor, and the feature point extracted in the second image is detected based on an image sensor of the moving camera.

18. The vibration correcting method of claim 17 , wherein the vibration comprises at least one of movements including user's handshake, rotation, in-plane rotation, three-axis rotation, translation, tilting, panning, and any other movement of the moving camera.

19. A vibration correcting method in a moving camera, the vibration correcting method comprising:
extracting a feature point in an input image of the moving camera;
predicting a position, to which a feature point extracted in a first image is to move due to vibration of the moving camera, by using a motion sensor;
generating a motion vector between the predicted position to which the feature point extracted in the first image is to move and a feature point extracted in a second image captured by the moving camera after occurrence of the vibration; and
correcting vibration of the second image based on the motion vector,
wherein the correcting the vibration the second image comprises determining that a position of the feature point extracted in the second image is accurate, when a distance between the predicted position to which the feature point extracted in the first image is to move and the feature point extracted in the second image captured by the moving camera after the occurrence of the vibration is within a predetermined range, and correcting the vibration of the second image by using position information of the feature point extracted in the second image.

Assignees

• Hanwha Techwin Co Ltd

Inventors

• Ryu Yeongeol
• Park Sangji
• Lee Hwalsuk

Classifications

• H04N23/683

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N23/667

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title

• H04N23/6812

  based on additional sensors, e.g. acceleration sensors · CPC title

• H04N23/681

  Motion detection · CPC title

• H04N23/685 Primary

  performed by mechanical compensation · CPC title