Apparatus and method for estimating camera pose

Patent metadata

Field	Value
Publication number	US-9916689-B2
Application number	US-201514611824-A
Country	US
Kind code	B2
Filing date	Feb 2, 2015
Priority date	Feb 17, 2014
Publication date	Mar 13, 2018
Grant date	Mar 13, 2018

Abstract

An apparatus for estimating a camera pose includes an image acquisition unit to acquire a photographed image, a motion sensor to acquire motion information of the apparatus for estimating the camera pose, a static area detector to detect a static area of the photographed image based on the photographed image and the motion information, and a pose estimator to estimate a camera pose based on the detected static area.

First claim

What is claimed is:

1. An apparatus for estimating a camera pose, the apparatus comprising:
an image acquisition unit configured to acquire a first image and a second image;
a motion sensor configured to acquire motion information of the apparatus;
a processor implemented static area detector configured to
detect a position of a first point of the first image and a position of a second point of the second image,
determine a predicted position of the second point based on the detected position of the first point and the motion information, and
determine an area of the second image to be a static area in response to a difference between the predicted position of the second point and the detected position of the second point being less than a threshold; and

a pose estimator configured to estimate the camera pose based on the determined static area.

2. The apparatus of claim 1 , wherein the first image is a first color image and the second image is a second color image.

3. The apparatus of claim 2 , wherein the pose estimator is configured to:
match corresponding points between the first image and the second image; and
estimate a rotation value from the motion information.

4. The apparatus of claim 3 , wherein the pose estimator is configured to calibrate the estimated rotation value to correspond to a disposition in which the camera is considered to be at a position identical to the motion sensor.

5. The apparatus of claim 3 , wherein the pose estimator is configured to:
calculate an essential matrix based on a matching relationship between the matched corresponding points and the estimated rotation value; and
estimate a translation value based on the calculated essential matrix.

6. The apparatus of claim 5 , wherein the static area detector is configured to determine the static area based on the estimated rotation value and the estimated translation value.

7. The apparatus of claim 6 , wherein the static area detector is configured to:
apply the estimated rotation value and the estimated translation value to the first point of the first image; and
predict the position of the second point of the second image.

8. An apparatus for estimating a camera pose, the apparatus comprising:
an image acquisition unit to acquire a photographed image;
a motion sensor to acquire motion information of the apparatus for estimating the camera pose;
a static area detector to detect a static area of the photographed image based on the photographed image and the motion information; and
a pose estimator to estimate the camera pose based on the detected static area,
wherein the image acquisition unit acquires a first color image and a second color image,
wherein the pose estimator matches corresponding points between the first color image and the second color image, estimates a rotation value from the motion information, calibrates the estimated rotation value to correspond to a disposition in which the camera is at a position identical to the motion sensor, calculates an essential matrix based on a matching relationship between the matched corresponding points and the estimated rotation value, estimates a translation value based on the calculated essential matrix,
wherein the static area detector applies the estimated rotation value and the estimated translation value to the corresponding point of the first color image, detects a predicted position of the corresponding point of the second color image, and detects an area in which a difference between the predicted position of the corresponding point of the second color image and an actual position of the corresponding point of the second color image is less than a predetermined threshold value to be the static area.

9. The apparatus of claim 1 , wherein the image acquisition unit is configured to acquire a first depth image and a second depth image respectively as the first image and the second image.

10. The apparatus of claim 9 , wherein the pose estimator is configured to select a pair of three-dimensional (3D) points from the first depth image and the second depth image that satisfies the motion information acquired from the motion sensor.

11. The apparatus of claim 10 , wherein the pose estimator is configured to estimate a translation value based on the selected pair of 3D points.

12. The apparatus of claim 11 , wherein the static area detector is configured to:
apply a rotation value and the translation value to a 3D point of the first depth image; and
estimate a predicted position of a 3D point of the second depth image.

13. The apparatus of claim 12 , wherein the rotation value is estimated from the motion information.

14. An apparatus for estimating a camera pose, the apparatus comprising:
an image acquisition unit to acquire a photographed image;
a motion sensor to acquire motion information of the apparatus for estimating the camera pose;
a static area detector to detect a static area of the photographed image based on the photographed image and the motion information; and
a pose estimator to estimate the camera pose based on the detected static area,
wherein the image acquisition unit acquires a first depth image and a second depth image,
wherein the pose estimator selects a pair of three-dimensional (3D) points from the first depth image and the second depth image that satisfies the motion information acquired from the motion sensor, and estimates a translation value based on the selected pair of 3D points, and
wherein the static area detector applies a rotation value and the translation value to a 3D point of the first depth image, and estimates a predicted position of a 3D point of the second depth image, and
wherein the static area detector detects an area in which a difference between the predicted position of the 3D point of the second depth image and an actual position of a 3D point of the second depth image is less than a predetermined threshold value to be the static area.

15. The apparatus of claim 14 , wherein the rotation value is estimated from the motion information.

16. A processor implemented method of controlling an apparatus for estimating a camera pose, the method comprising:
detecting a position of a first point of an acquired first image and a position of a second point of an acquired second image;
determining a predicted position of the second point based on the detected position of the first point and acquired motion information from a motion sensor;
determining an area of the second image to be a static area in response to a difference between the predicted position of the second point and the detected position of the second point being less than a threshold; and
estimating the camera pose based on the determined static area.

17. The method of claim 16 , wherein the
acquired first image and the acquired second image are
an image pair respectively as a first color image and a second color image, or
a depth image pair respectively as a first depth image and a second depth image.

18. The method of claim 17 , wherein the estimating of the camera pose is based on:
corresponding points of the first color image and the second color image, or
a pair of three-dimensional (3D) points from the static area based on the first depth image and the second depth image.

19. The method of claim 16 , further comprising:
outputting a control signal corresponding to the estimated camera pose.

20. The method of claim 19 , further comprising:
executing an application that provides an augmented reality (AR) service,

Assignees

• Samsung Electronics Co Ltd

Inventors

• Rhee Seon Min
• Park Seung In
• Lee Yong Beom
• Lee Hyong Euk

Classifications

• G06T2200/04

  involving 3D image data · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T2207/10012

  Stereo images · CPC title

• G06T7/73

  using feature-based methods · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title