Method and apparatus for estimating motion in video, method and apparatus for stabilizing video, and computer-readable recording medium

Patent metadata

Field	Value
Publication number	US-10044936-B2
Application number	US-201615065167-A
Country	US
Kind code	B2
Filing date	Mar 9, 2016
Priority date	Sep 16, 2015
Publication date	Aug 7, 2018
Grant date	Aug 7, 2018

Abstract

A method of stabilizing a video is provided. The method includes detecting a feature point from a first frame; predicting a location of the feature point in a second frame based on a location of the feature point in the first frame and a predetermined parameter; detecting the feature point from the second frame; determining a distance between the predicted location and a location of the feature point detected from the second frame; and updating the parameter based on a location difference between the feature point detected in the first frame and the feature point detected in the second frame, in response to determining that the distance is within a predetermined threshold.

First claim

What is claimed is:

1. A method performed by an apparatus for stabilizing a video, the method comprising:
detecting a feature point from a first frame;
predicting a location of the feature point in a second frame based on a location of the feature point in the first frame and a predetermined parameter;
detecting the feature point from the second frame;
determining a distance between the predicted location of the feature point in the second frame and a location of the feature point detected from the second frame; and
updating the predetermined parameter based on a location difference between the feature point detected in the first frame and the feature point detected in the second frame, in response to determining that the distance is within a predetermined threshold,
wherein the video is recorded by a camera which is rotatable in at least one axis direction,
wherein the predetermined parameter indicates a relationship between an angular velocity of the camera, which is measured between a time point when the first frame is captured and a time point when the second frame is captured, and pixel movement between the first frame and the second frame, and
wherein the predicting comprises:
acquiring the angular velocity of the camera, the angular velocity being sensed by a gyro sensor;
estimating the location difference by multiplying the angular velocity by the predetermined parameter; and
predicting the location of the feature point in the second frame based on the location of the feature point detected from the first frame and the estimated location difference.

2. The method of claim 1 , wherein the updating comprises updating the predetermined parameter based on the angular velocity.

3. The method of claim 2 , wherein the updating comprises:
calculating a new parameter based on the location difference and the angular velocity; and
updating the predetermined parameter with the new parameter.

4. The method of claim 2 , wherein the updating comprises updating the predetermined parameter based on a ratio of the location difference to the angular velocity.

5. The method of claim 2 , wherein the updating comprises:
calculating an average of a preset number of recent parameters which are calculated based on the location difference and the angular velocity at different time points; and
updating the predetermined parameter with the average of the preset number of recent parameters.

6. The method of claim 1 , wherein the acquiring comprises receiving a plurality of instantaneous angular velocities of the camera that are consecutively sensed between a time point when the first frame is captured and a time point when the second frame is captured, and
wherein the determining a location difference comprises calculating an average angular velocity of the plurality of instantaneous angular velocities and multiplying the average angular velocity by the predetermined parameter.

7. The method of claim 1 , wherein the predetermined parameter is dependent on a focal length of the camera.

8. The method of claim 1 , wherein the detecting of the feature point from the first frame, the predicting, the detecting of the feature point from the second frame, the determining, and the updating are performed regarding a plurality of feature points comprising the feature point.

9. The method of claim 8 , wherein the updating comprises estimating the location difference based on a vector defined by the location of the feature point detected from the second frame and the location of the feature point detected from the first frame.

10. The method of claim 1 , further comprising correcting the video based on the location difference.

11. An apparatus for stabilizing a video, the apparatus comprising:
at least one processor configured to implement:
a feature point detector configured to detect a feature point from a first frame;
a feature point location predictor configured to predict a location of the feature point in a second frame based on a location of the feature point in the first frame and a predetermined parameter;
a feature point tracker configured to detect the feature point from the second frame;
a feature point verifier configured to determine a distance between the predicted location of the feature point in the second frame and a location of the feature point detected from the second frame;
a motion estimator configured to determine a location difference between the feature point detected in the first frame and the feature point detected in the second frame; and
an updater configured to update the predetermined parameter based on the location difference in response to the feature point verifier determining that the distance is within a predetermined threshold,

wherein the video is recorded by a camera which is rotatable in at least one axis direction,
wherein the predetermined parameter indicates a relationship between an angular velocity of the camera, which is measured between a time point when the first frame is captured and a time point when the second frame is captured, and pixel movement between the first frame and the second frame, and
wherein the feature point location predictor is further configured to:
acquire the angular velocity of the camera, the angular velocity being sensed by a gyro sensor;
estimate the location difference by multiplying the angular velocity by the predetermined parameter; and
predict the location of the feature point in the second frame based on the location of the feature point detected from the first frame and the estimated location difference.

12. The apparatus of claim 11 , wherein the at least one processor is further configured to implement a motion corrector configured to correct the video based on the location difference.

13. A non-transitory computer-readable recording medium having recorded thereon a program for executing the method of claim 1 .

Assignees

• Hanwha Techwin Co Ltd

Inventors

• Ryu Yeongeol
• Park Sangji
• Cha Junho
• Noh Seungin

Classifications

• H04N23/6812

  based on additional sensors, e.g. acceleration sensors · CPC title

• H04N23/6811 Primary

  based on the image signal · CPC title

• H04N23/683 Primary

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N23/6815

  by distinguishing pan or tilt from motion · CPC title

• H04N5/23258

  Electricity · mapped topic