Online compensation of thermal distortions in a stereo depth camera

Patent metadata

Field	Value
Publication number	US-9813692-B2
Application number	US-201514865196-A
Country	US
Kind code	B2
Filing date	Sep 25, 2015
Priority date	Sep 25, 2015
Publication date	Nov 7, 2017
Grant date	Nov 7, 2017

Abstract

A method and apparatus for performing temperature compensation for thermal distortions in a camera system. In one embodiment, the system comprises a first camera configured to capture a sequence of images of the object; a second device; a processing unit to receive the sequence of images and determine depth information in response to parameters of the camera and the second device; one or more temperature sensors; and a thermal correction unit responsive to temperature information from the one or more temperature sensors to adjust one or more of the calibration parameters of the first camera and the second device.

First claim

We claim:

1. An apparatus comprising:
a projector configured to project a sequence of light patterns on an object;
a first camera configured to capture a sequence of images of the object illuminated with the projected light patterns;
one or more temperature sensors;
a thermal correction unit responsive to temperature information from the one or more temperature sensors to adjust one or more of calibration parameters of the first camera and the projector; and
a processing unit to receive the sequence of images and determine depth information in response to the one or more adjusted calibration parameters of the camera and the projector, wherein the processing unit is configured to determine the depth information with a triangulation process using the one or more adjusted calibration parameters.

2. The apparatus defined in claim 1 wherein the calibration parameters characterize an optical system formed by the projector and the first camera and comprise intrinsic parameters of the first camera and the projector and extrinsic parameters of first camera.

3. The apparatus defined in claim 2 wherein the intrinsic parameters comprise one or more of a focal distance of the first camera, a principal point of projection of the projector, and a distortion model and wherein the extrinsic parameters comprise a relative location and orientation of the projector with respect to the first camera.

4. The apparatus defined in claim 1 wherein the calibration parameters are initialized by one or more calibration procedures and further wherein the thermal correction unit is operable to adjust one or more of the calibration parameters based on outputs from a thermal correction model that is responsive to the temperature information from the one or more temperature sensors.

5. The apparatus defined in claim 1 wherein the first camera comprises a digital infrared camera.

6. The apparatus defined in claim 1 wherein the projector is configured to project one-dimensional coded light patterns on the object.

7. The apparatus defined in claim 1 further comprising a second camera configured to capture a second image of the object illuminated with the projected light patterns and configured to capture a texture image of the object in the second image.

8. The apparatus defined in claim 7 wherein the first camera comprises an IR camera, and the second camera comprises a red, green, and blue (RGB) camera.

9. A method comprising:
projecting, using a projector, a sequence of light patterns on an object;
capturing, using a first camera, a sequence of images of the object illuminated with the projected light patterns;
adjusting one or more of calibration parameters of the first camera and the projector responsive to temperature information from the one or more temperature sensors; and
determining depth information using the one or more adjusted calibration parameters of the camera and the projector, wherein determining depth information comprises performing a triangulation process using the one or more adjusted calibration parameters.

10. The method defined in claim 9 wherein the calibration parameters characterize an optical system formed by the projector and the first camera and comprise intrinsic parameters of the first camera and the projector and extrinsic parameters of first camera.

11. The method defined in claim 10 wherein the intrinsic parameters comprise one or more of a focal distance of the first camera, a principal point of projection of the projector, and a distortion model and wherein the extrinsic parameters comprise a relative location and orientation of the projector with respect to the first camera.

12. The method defined in claim 9 further comprising projecting, using a projector, a sequence of light patterns on an object, and wherein the calibration parameters are initialized by one or more calibration procedures and further wherein adjusting the one or more of the calibration parameters of the projector and the camera is based on outputs from a thermal correction model that is responsive to the temperature information from the one or more temperature sensors.

13. An article of manufacture having one or more non-transitory computer readable storage media storing instructions which when executed by a system to perform a method comprising:
projecting a sequence of light patterns on an object;
capturing a sequence of images of the object illuminated with the projected light patterns;
adjusting, based on temperature information from one or more temperature sensors, one or more of calibration parameters of the projector and the camera;
determining depth information in response to the one or more adjusted parameters of the projector and camera, wherein determining depth information comprises performing a triangulation process using the one or more adjusted calibration parameters.

14. The article of manufacture defined in claim 13 wherein the calibration parameters characterize an optical system formed by the projector and the first camera and comprise intrinsic parameters of the first camera and the projector and extrinsic parameters of first camera.

15. The article of manufacture defined in claim 14 wherein the intrinsic parameters comprise one or more of a focal distance of the first camera, a principal point of projection of the projector, and a distortion model and wherein the extrinsic parameters comprise a relative location and orientation of the projector with respect to the first camera.

16. The article of manufacture defined in claim 13 wherein the calibration parameters are initialized by one or more calibration procedures and further wherein adjusting the one or more of the calibration parameters of the projector and the camera is based on outputs from a thermal correction model that is responsive to the temperature information from the one or more temperature sensors.

Assignees

• Intel Corp

Inventors

• Zabatani Aviad
• Bareket Sagy
• Menashe Ohad
• Sperling Erez
• Bronstein Alex
• Bronstein Michael
• Kimmel Ron
• Surazhsky Vitaly

Classifications

• H04N13/246 Primary

  Calibration of cameras · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• H04N13/243

  using three or more two-dimensional [2D] image sensors · CPC title

• G06T7/85

  Stereo camera calibration · CPC title