Method and apparatus for calibrating image

Patent metadata

Field	Value
Publication number	US-2016150211-A1
Application number	US-201514932405-A
Country	US
Kind code	A1
Filing date	Nov 4, 2015
Priority date	Nov 20, 2014
Publication date	May 26, 2016
Grant date	—

Abstract

An apparatus for calibrating a multiview image may extract feature points from the multiview image and perform image calibration based on the extracted feature points, track corresponding feature points in temporally successive image frames of a first view image, and perform the image calibration based on pairs of corresponding feature points between the feature points tracked from the first view image and feature points of a second view image.

First claim

What is claimed is:

1 . A method of calibrating an image, the method comprising:
extracting feature points from a multiview image, the multiview image including a first view image and a second view image; selecting first feature points, corresponding to feature points of a preceding image frame of the first view image, from among feature points in a current image frame of the first view image; selecting second feature points, corresponding to feature points in a current image frame of the second view image, from among the first feature points; and performing image calibration on the current image frame of the second view image based on the second feature points.

2 . The method of claim 1 , wherein the selecting of the first feature points comprises:
determining pairs of corresponding feature points between the current image frame of the first view image and the preceding image frame of the first view image; and selecting the first feature points from among the pairs of the feature points using a fundamental matrix between the current image frame of the first view image and the preceding image frame of the first view image.

3 . The method of claim 2 , wherein the selecting of the first feature points comprises:
randomly selecting a number of pairs of feature points from among the selected pairs of the feature points; and determining the fundamental matrix based on location information of the selected number of pairs of the feature points.

4 . The method of claim 2 , wherein the selecting of the pairs of the feature points comprises:
determining the pairs of the feature points based on a difference in a feature point descriptor indicating a neighborhood property of a feature point.

5 . The method of claim 1 , wherein the performing of the image calibration comprises:
performing the image calibration on the current image frame of the second view image based on the second feature points and third feature points of the current image frame of the second view image corresponding to the second feature points.

6 . The method of claim 5 , wherein the performing of the image calibration comprises:
determining a homography between the current image frame of the first view image and the current image frame of the second view image based on location information of the second feature points and location information of the third feature points; and adjusting locations of pixels in the current image frame of the second view image by applying the homography to the current image frame of the second view image.

7 . The method of claim 5 , wherein the performing of the image calibration comprises:
filtering the second feature points based on a search region; and performing the image calibration on the current image frame of the second view image based on ones of the second feature points remaining after the filtering.

8 . The method of claim 5 , wherein the performing of the image calibration comprises:
adjusting vertical coordinate values of pixels in the current image frame of the second view image based on location information of the second feature points and location information of the third feature points.

9 . The method of claim 1 , further comprising:
adjusting color values of pixels in the current image frame of the second view image based on a homography between the current image frame of the first view image and the current image frame of the second view image.

10 . The method of claim 9 , wherein the adjusting of the color values comprises:
warping the current image frame of the second view image using the homography; mapping the warped current image frame of the second view image to the current image frame of the first view image; identifying corresponding pixels between the current image frame of the second view image and the current image frame of the first view image mapped with the current image frame of the second view image; determining a color transform matrix based on color values of the identified pixels; and adjusting the color values of the pixels in the current image frame of the second view image by applying the color transform matrix to the current image frame of the second view image.

11 . The method of claim 1 , wherein the extracting of the feature point comprises:
downsampling the current image frame of the first view image; and extracting a feature point from the downsampled current image frame.

12 . The method of claim 1 , wherein the multiview image comprises a third view image.

13 . A non-transitory computer-readable medium comprising program code that, when executed by a processor, performs the method of claim 1 .

14 . An apparatus for calibrating an image, the apparatus comprising:
a feature point extractor configured to extract feature points from a multiview image, the multiview image including a first view image and a second view image; a feature point determiner configured to select feature points to be used for image calibration from a current image frame of the first view image and a current image frame of the second view image based on feature points in a preceding image frame of the first view image; and an image calibrator configured to perform the image calibration based on the selected feature points.

15 . The apparatus of claim 14 , wherein the feature point determiner is configured to select first feature points corresponding to the feature points of the preceding image frame from among feature points in the current image frame of the first view image, and select, as the feature points to be used for the image calibration, second feature points corresponding to feature points in the current image frame of the second view image from among the first feature points.

16 . The apparatus of claim 14 , wherein the feature point determiner is configured to determine pairs of corresponding feature points between the current image frame and the preceding image frame of the first view image, and select the first feature points from the pairs of the feature points using a fundamental matrix between the current image frame and the preceding image frame of the first view image.

17 . The apparatus of claim 14 , wherein the image calibrator comprises:
a geometric calibrator configured to adjust locations of pixels in the current image frame of the second view image based on location information of the feature points to be used for the image calibration.

18 . The apparatus of claim 17 , wherein the geometric calibrator is configured to determine a homography between the current image frame of the first view image and the current image frame of the second view image based on the location information of the feature points to be used for the image calibration, and adjust the locations of the pixels in the current image frame of the second view image by applying the homography to the current image frame of the second view image.

19 . The apparatus of claim 14 , wherein the image calibrator comprises:
a color corrector configured to adjust color values of pixels in the current image frame of the second view image based on a homography between the current image frame of the first view image and the current image frame of the second view image.

20 . The apparatus of claim 19 , wherein the color corrector is configured to determine a color transform matrix based on the homography, and adjust color values of pixels in a current image frame of the second view image by applying the color transform matrix to the current image frame of th

Assignees

• Samsung Electronics Co Ltd

Inventors

• Hwang Hyoseok
• Cho Yang Ho
• Nam Dong Kyung
• Choi Ouk

Classifications

• H04N13/204

  using stereoscopic image cameras (stereoscopic photography G03B35/00) · CPC title

• H04N13/25

  using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor · CPC title

• G06T2207/10012

  Stereo images · CPC title

• G06T2207/20024

  Filtering details · CPC title

• H04N13/257

  Colour aspects · CPC title