System and method for image capture device pose estimation

Patent metadata

Field	Value
Publication number	US-2017249751-A1
Application number	US-201715440092-A
Country	US
Kind code	A1
Filing date	Feb 23, 2017
Priority date	Feb 25, 2016
Publication date	Aug 31, 2017
Grant date	—

Abstract

A method for estimating a plurality of camera, comprising using at least one processor executing a code for: extracting a plurality of image features of a plurality of landmarks from a plurality of images captured by at least one camera from at least one pose, the plurality of landmarks calibrated with respect to a certain coordinate system; identifying among the plurality of image features at least one image feature documented in at least some of the images; producing scale values of at least one common image feature by analyzing the at least some of the images; determining a plurality of estimated poses of the at least one camera with respect to the certain coordinate system by using the scale values in calculating a minimal re-projection error between the plurality of image features and a plurality of predicted image features; and outputting the plurality of estimated poses.

First claim

What is claimed is:

1 . A method for estimating a plurality of image capture device poses, comprising:
using at least one hardware processor executing a code for:
extracting a plurality of observed image features of a plurality of landmarks from a plurality of images captured by at least one image capture device from at least one pose, said plurality of landmarks calibrated with respect to a certain coordinate system;
identifying among said plurality of observed image features at least one observed image feature documented in at least some of said images;
producing image feature scale values of said at least one common observed image feature by analyzing said at least some of said images;
determining a plurality of estimated poses of said at least one image capture device with respect to said certain coordinate system by using said image feature scale values in calculating a minimal re-projection error between said plurality of observed image features and a plurality of predicted image features; and
outputting said plurality of estimated poses.

2 . The method of claim 1 , wherein said determining a plurality of estimated poses of said at least one image capture device with respect to said certain coordinate system comprises using a scale re-projection error term in a bundle adjustment optimization calculation;
wherein said scale re-projection error term is computed using said image feature scale values.

3 . The method of claim 2 , wherein said bundle adjustment optimization calculation comprises at least one cost function;
wherein said at least one cost function comprises at least one scale error term expressing a scale re-projection error using said image feature scale information; and wherein said bundle adjustment optimization calculation comprises:
producing a plurality of estimated image capture device pose sets, each of said plurality of sets comprising a plurality of estimated image capture device poses, each of said plurality of estimated image capture device poses associated with one of said plurality of images;
producing a plurality of estimated costs by applying said at least one cost function to each of said plurality of estimated capture device pose sets;
identifying a minimum estimated cost of said plurality of estimated costs; and
selecting the estimated capture device pose set used to produce said minimum estimated cost.

4 . The method of claim 3 , wherein said producing a plurality of estimated image capture device pose sets comprises applying sparsity-aware iterative optimization techniques;
wherein said sparsity-aware iterative optimization techniques are members of a group comprising: g2o, Georgia Tech smoothing and mapping (gtsam) and Ceres Solver.

5 . The method of claim 4 , wherein said at least one scale error term is computed;
wherein computing said at least one scale error term comprises computing a difference between a measured image feature scale and a predicted feature scale; wherein said measured image feature scale is calculated by a feature detector; and wherein said predicted feature scale is computed using an image capture device focal length, a distance from said common image feature's landmark to one of said plurality of estimated camera poses and a size estimate of said common image feature's landmark.

6 . The method of claim 5 ,
wherein computing said predicted feature scale comprises:
dividing said size estimate by said distance to produce a ratio; and
multiplying said ratio by said focal length; and
wherein said computing a difference between a measured image feature scale and a predicted feature scale comprises subtracting said predicted feature scale from said image feature scale.

7 . The method of claim 4 , wherein said feature detector is Scale Invariant Feature Transform (SIFT).

8 . The method of claim 4 , wherein computing said difference comprises calculating a Squared Mahalanobis Distance between said measured image feature scale and said predicted feature scale, having a measurement covariance matrix of a non zero-mean Gaussian distribution of noise elements in said image feature scale values.

9 . The method of claim 8 , further comprising producing said measurement covariance matrix by computing a covariance matrix of a zero-mean Gaussian distribution of said noise elements in said image feature scale values using methods as known to the art.

10 . The method of claim 1 , wherein said identifying among said plurality of observed image features at least one observed image feature is by applying image matching algorithms.

11 . The method of claim 10 , wherein said image matching algorithms include Scale Invariant Feature Transform (SIFT) and Random Sample Consensus (RANSAC).

12 . The method of claim 1 , wherein said image feature scale values are produced while said identifying among said plurality of observed image features at least one observed image feature.

13 . The method of claim 1 , wherein said certain coordinate system is a world coordinate system.

14 . The method of claim 1 , wherein the origin of said certain coordinate system is calibrated to a pose of one of said at least one image capture devices when capturing a first image of said plurality of images.

15 . A system for estimating a plurality of image capture device poses, comprising:
at least one hardware processor executing a code for:
extracting a plurality of observed image features of a plurality of landmarks from a plurality of images captured by at least one image capture device from at least one pose, said plurality of landmarks calibrated with respect to a certain coordinate system;
identifying among said plurality of observed image features at least one observed image feature documented in at least some of said images;
producing image feature scale values of said at least one common observed image feature by analyzing said at least some of said images;
determining a plurality of estimated poses of said at least one image capture device with respect to said certain coordinate system by using said image feature scale values in calculating a minimal re-projection error between said plurality of observed image features and a plurality of predicted image features; and
outputting said plurality of estimated poses.

16 . The system of claim 15 , further comprising at least one digital storage hardware, electrically connected to said at least one hardware processor.

17 . The system of claim 15 , further comprising at least one non-volatile data storage;
wherein said at least one hardware processor is connected to said at least one non-volatile data storage via a network.

18 . The system of claim 17 , wherein said non-volatile data storage is a cloud database.

19 . The system of claim 15 , wherein said image capture device is installed on a moving arm.

20 . The system of claim 15 , wherein said image capture device is installed on a vehicle.

21 . The system of claim 20 , wherein said vehicle is a member of a group including: an airplane, a drone, a helicopter, a floating vehicle, a motorized vehicle, a car, a bicycle, a motorcycle, an electric vehicle, a balloon, a blimp and a cart.

Assignees

• Technion Res & Dev Foundation

Inventors

• Indelman Vadim

Classifications

• G06T2207/30244

  Camera pose · CPC title

• G06T2207/30248

  Vehicle exterior or interior · CPC title

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06T7/579 Primary

  from motion · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title