Automated camera array calibration

Patent metadata

Field	Value
Publication number	US-9846960-B2
Application number	US-201213566877-A
Country	US
Kind code	B2
Filing date	Aug 3, 2012
Priority date	May 31, 2012
Publication date	Dec 19, 2017
Grant date	Dec 19, 2017

Abstract

The automated camera array calibration technique described herein pertains to a technique for automating camera array calibration. The technique can leverage corresponding depth and single or multi-spectral intensity data (e.g., RGB (Red Green Blue) data) captured by hybrid capture devices to automatically determine camera geometry. In one embodiment it does this by finding common features in the depth maps between two hybrid capture devices and derives a rough extrinsic calibration based on shared depth map features. It then uses the intensity (e.g., RGB) data corresponding to the depth maps and uses the features of the intensity (e.g., RGB) data to refine the rough extrinsic calibration.

First claim

What is claimed is:

1. A computer-implemented process for calibrating an array of capture devices, comprising the process actions of:
using hybrid capture devices, which capture both intensity data and depth data, to simultaneously capture depth maps and corresponding intensity images of a scene;
finding common features in the depth maps from two hybrid capture devices;
automatically computing a rough calibration of extrinsic parameters of the hybrid capture devices, comprising both rotations and translations which detail the spatial location of each of the hybrid capture devices as well as the direction that the hybrid capture device is pointing using the shared depth map features.

2. The computer-implemented process of claim 1 , further comprising the process actions of:
if the hybrid capture devices are not temporally synchronized,
separating the moving and non-moving data of the scene in the depth images and intensity images of the scene; and
using only the non-moving data of the scene for finding the common features.

3. The computer-implemented process of claim 1 wherein the depth maps are down sampled prior to finding the common features.

4. The computer-implemented process of claim 1 , wherein once the rough calibration is found, the intensity data is used to refine the rough calibration.

5. The computer-implemented process of claim 4 wherein the intensity data is downsampled prior to using the RGB data to refine the rough calibration.

6. The computer-implemented process of claim 3 , wherein the relationship between the corresponding depth map and intensity image data is used to refine the rough calibration.

7. The computer-implemented process of claim 1 , wherein if a rough calibration cannot be computed,
manually scanning the scene with one capture device by hand to capture the scene,
building a model of the scene using images captured of the scene captured by the one capture device,
computing a rough calibration by comparing the depth map images captured by each camera against the created model.

8. The computer-implemented process of claim 1 wherein separating the moving and non-moving data further comprises employing an optical flow diagram.

9. A computer-implemented process for calibrating an array of capture devices, comprising the process actions of:
using hybrid capture devices, which capture both intensity data and depth data, to simultaneously capture depth maps and corresponding intensity images of a scene;
finding common features in the depth maps from two hybrid capture devices;
automatically computing a rough calibration of extrinsic parameters of the hybrid capture devices; and
using the intensity data to refine the rough calibration by,
for an intensity image captured using a current hybrid capture device,
identifying features in the intensity image;
projecting each feature onto an intensity image of another hybrid capture device using the rough calibration data obtained from matching the depth maps; and
refining the rough calibration by matching the intensity features in the intensity image captured by the current hybrid capture device to intensity features in the intensity image captured by the other hybrid capture device using feature or window based matching.

10. The computer-implemented process of claim 9 further comprising checking the refined calibration by verifying matching intensity feature points and discarding non-matching feature points.

11. The computer-implemented process of claim 10 further comprising sending the non-discarded features to a bundle adjustment algorithm to calibrate the hybrid capture devices.

12. The computer-implemented process of claim 9 wherein the depth map data is obtained by using two IR cameras in a given hybrid capture device, and the intensity data is obtained by a RGB camera in the same given hybrid capture device.

13. The computer-implemented process of claim 9 wherein the rough calibration and the refined calibration are repeated using progressively less down sampled depth maps and RGB images.

Assignees

• Kirk Adam G
• Eshet Yaron
• Eraker David
• Microsoft Technology Licensing Llc

Inventors

• Kirk Adam G
• Eshet Yaron
• Eraker David

Classifications

• H04N13/257

  Colour aspects · CPC title

• G06T15/08

  Volume rendering · CPC title

• G06T2210/56

  Particle system, point based geometry or rendering · CPC title

• H04N7/157

  defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827) · CPC title

• H04N7/15

  Conference systems · CPC title