Camera pose estimation apparatus and method

Patent metadata

Field	Value
Publication number	US-9875545-B2
Application number	US-201414903982-A
Country	US
Kind code	B2
Filing date	May 15, 2014
Priority date	Jul 9, 2013
Publication date	Jan 23, 2018
Grant date	Jan 23, 2018

Abstract

Provided is a camera pose estimation apparatus that estimates an initial camera pose using one of an input depth image and an input color image, and refines the initial camera pose using the other image. When the initial camera pose is estimated using the input depth image, the radius of a first area, in which color information is matched for refinement, can be adaptively set according to the distribution of the depth value of at least one first point that is subject to matching.

First claim

What is claimed is:

1. An apparatus for camera pose estimation, the apparatus comprising:
an estimator configured to estimate an initial camera pose associated with a first image by matching at least one first point included in a first frame and at least one second point included in a second frame, with respect to the first image that is one of an input depth image and an input color image;
a determiner to determine whether to use the input color image to estimate the initial camera pose based on a reliability of the input color image and a reliability of the input depth image; and
a refiner configured to provide a refined camera pose by refining the initial camera pose using a second image, between the input depth image and the input color image.

2. The apparatus of claim 1 , wherein the estimator is further configured to match the at least one first point and the at least one second point using an iterative closest point (ICP) algorithm.

3. The apparatus of claim 1 , wherein the first image corresponds to the input depth image and the second image corresponds to the input color image.

4. The apparatus of claim 3 , wherein the estimator is further configured to select the at least one first point included in the first frame and the at least one second point included in the second frame, from among point clouds included in the input depth image.

5. The apparatus of claim 4 , wherein the estimator is further configured to select, from the first frame and the second frame, at least one point having a depth variation less than or equal to a threshold variation among the point clouds.

6. The apparatus of claim 3 , wherein the refiner is further configured to refine the initial camera pose based on the input color image within a first region included in a first radius around the at least one first point.

7. The apparatus of claim 6 , wherein the first radius is set to increase according to an increase in a depth value of the at least one first point.

8. An apparatus for camera pose estimation, the apparatus comprising:
a determiner configured to determine a camera pose estimation mode associated with an input depth image and an input color image by analyzing at least one feature of the input depth image and the input color image and to determine whether to use the input color image to estimate an initial camera pose based on a reliability of the input color image and a reliability of the input depth image;
an estimator configured to estimate the initial camera pose between a first frame and a second frame by matching at least one first point included in the first frame and at least one second point included in the second frame different from the first frame, with respect to the input depth image when the determined camera pose estimation mode corresponds to a camera pose estimation using all of the input depth image and the input color image; and
a refiner configured to provide a refined camera pose by refining the initial camera pose through matching between surrounding color information of the at least one first point and surrounding color information of the at least one second point using the input color image.

9. The apparatus of claim 8 , wherein the determiner is further configured to determine that the camera pose estimation mode corresponds to the camera pose estimation using all of the input depth image and the input color image, in response to a depth feature point extracted from a depth value of the input depth image being greater than or equal to a first threshold, and a color feature point extracted from a color value of the input color image being greater than or equal to a second threshold.

10. The apparatus of claim 8 , wherein the refiner is further configured to refine the initial camera pose based on the input color image within a first region included in a first radius around the at least one first point.

11. The apparatus of claim 10 , wherein the first radius is set to increase according to an increase in a depth value of the at least one first point.

12. A method for camera pose estimation, the method comprising:
estimating, by an estimator of a camera pose estimation apparatus, an initial camera pose associated with an input depth image by primarily matching at least one first point included in a first frame and at least one second point included in a second frame, with respect to the input depth image using an iterative closest point (ICP) algorithm;
determining, by a determiner, whether to use the input color image to estimate the initial camera pose based on a reliability of the input color image and a reliability of the input depth image; and
establishing, by a refiner of the camera pose estimation apparatus, a first radius of a first region including the at least one first point in order to perform secondary matching using an input color image associated with the input depth image, the secondary matching using a color feature matching within the first region.

13. The method of claim 12 , wherein the first radius is set to increase according to an increase in a depth value of the at least one first point.

14. The method of claim 12 , further comprising:
determining, by the determiner of the camera pose estimation apparatus, a camera mode estimation mode based on depth feature information extracted from the input depth image and color feature information extracted from the input color image.

15. The method of claim 12 , wherein each of the at least one first point and the at least one second point corresponds to a point having a reliability greater than or equal to a predetermined threshold based on a depth distribution.

16. A non-transitory computer-readable medium comprising a program for instructing a computer to perform the camera pose estimation method according to claim 12 .

Assignees

• Samsung Electronics Co Ltd

Inventors

• Rhee Seon Min
• Lee Hyong Euk
• Lee Yong Beom

Classifications

• H04N23/60

  Control of cameras or camera modules · CPC title

• G06T7/80 Primary

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G06T7/38

  Registration of image sequences · CPC title

• G06T7/85

  Stereo camera calibration · CPC title

• G06T2207/30244

  Camera pose · CPC title