Online compensation of thermal distortions in a stereo depth camera

Patent metadata

Field	Value
Publication number	US-11438569-B2
Application number	US-202016983640-A
Country	US
Kind code	B2
Filing date	Aug 3, 2020
Priority date	Sep 25, 2015
Publication date	Sep 6, 2022
Grant date	Sep 6, 2022

Abstract

An example apparatus includes: a camera to record an image; memory to store instructions; and a processor in circuit with the memory, the processor to execute the instructions to: determine a depth based on: (a) the image and (b) a calibration parameter of the camera; and adjust the calibration parameter based on a temperature of the camera and the depth.

First claim

We claim:

1. An apparatus comprising:
means for modifying a calibration parameter of a capture device based on a compensation value, the compensation value corresponding to a temperature of the capture device; and
means for generating a depth value based on a pixel location of an object in an image captured by the capture device, a location of a projected plane in the image, and the calibration parameter, the depth value corresponding to a distance to the object in the image.

2. The apparatus of claim 1 , wherein the means for generating the depth value is to generate the depth value based on an intersection of the projected plane and a ray projected by the capture device.

3. The apparatus of claim 1 , further including means for generating the projected plane in the image by projecting at least one of a one-dimensional coded light pattern or a two-dimensional coded light pattern.

4. The apparatus of claim 1 , wherein the means for generating the depth value is to generate the depth value based on at least one of an intrinsic parameter or an extrinsic parameter of the capture device.

5. The apparatus of claim 4 , wherein the intrinsic parameter is a focal distance of the capture device, a principal point of projection of a projector that is to generate the projected plane, a distortion model, or a characteristic of an optical system formed by the projector and the capture device.

6. The apparatus of claim 4 , wherein the extrinsic parameter is at least one of a relative location of a projector or an orientation of the projector with respect to a camera of the capture device, the camera to capture the image, the projector to generate the projected plane.

7. The apparatus of claim 1 , wherein the means for modifying the calibration parameter is to modify the calibration parameter for ones of frames corresponding to a frame rate of the capture device.

8. The apparatus of claim 1 , wherein the pixel location is a first pixel location, the image is a first image, and the capture device includes a first camera to capture the first image, the means for generating the depth value to generate the depth value based on (1) the first pixel location, (2) a second pixel location of the object in a second image captured by a second camera of the capture device, and (3) the calibration parameter, the depth value corresponding to the distance to the object based on the first image and the second image.

9. A capture device comprising:
memory;
instructions; and
processor circuitry to execute the instructions to:
modify a calibration parameter of the capture device based on a compensation value, the compensation value corresponding to a temperature of the capture device; and
generate a depth value based on a pixel location of an object in an image captured by the capture device, a location of a projected plane in the image, and the calibration parameter, the depth value corresponding to a distance to the object in the image.

10. The capture device of claim 9 , wherein the processor circuitry is to execute the instructions to generate the depth value based on an intersection of the projected plane and a ray projected by the capture device.

11. The capture device of claim 9 , wherein the processor circuitry is to execute the instructions to generate the projected plane in the image by projecting at least one of a one-dimensional coded light pattern or a two-dimensional coded light pattern.

12. The capture device of claim 9 , wherein the processor circuitry is to execute the instructions to generate the depth value based on at least one of an intrinsic parameter or an extrinsic parameter of the capture device.

13. The capture device of claim 12 , wherein the intrinsic parameter is a focal distance of the capture device, a principal point of projection of a projector that is to generate the projected plane, a distortion model, or a characteristic of an optical system formed by the projector and the capture device.

14. The capture device of claim 12 , wherein the extrinsic parameter is at least one of a relative location of a projector or an orientation of the projector with respect to a camera of the capture device, the camera to capture the image, the projector to generate the projected plane.

15. The capture device of claim 9 , wherein the processor circuitry is to execute the instructions to modify the calibration parameter for ones of frames corresponding to a frame rate of the capture device.

16. The capture device of claim 9 , wherein the pixel location is a first pixel location, the image is a first image, and the capture device includes a first camera to capture the first image, the processor circuitry to execute the instructions to generate the depth value based on (1) the first pixel location, (2) a second pixel location of the object in a second image captured by a second camera of the capture device, and (3) the calibration parameter, the depth value corresponding to the distance to the object based on the first image and the second image.

17. A method comprising:
modifying, by executing an instruction with a processor, a calibration parameter of a capture device based on a compensation value, the compensation value corresponding to a temperature of the capture device; and
generating, by executing an instruction with the processor, a depth value based on a pixel location of an object in an image captured by the capture device, a location of a projected plane in the image, and the calibration parameter, the depth value corresponding to a distance to the object in the image.

18. The method of claim 17 , wherein the generating of the depth value is based on an intersection of the projected plane and a ray projected by the capture device.

19. The method of claim 17 , wherein the generating of the depth value is based on at least one of an intrinsic parameter or an extrinsic parameter of the capture device, the intrinsic parameter is a focal distance of the capture device, a principal point of projection of a projector that is to generate the projected plane, a distortion model, or a characteristic of an optical system formed by the projector and the capture device, the extrinsic parameter is at least one of a relative location of the projector or an orientation of the projector with respect to a camera of the capture device, wherein the camera is to capture the image, and the projector is to generate the projected plane.

20. The method of claim 17 , wherein the pixel location is a first pixel location, the image is a first image, and the capture device includes a first camera to capture the first image, the generating of the depth value based on (1) the first pixel location, (2) a second pixel location of the object in a second image captured by a second camera of the capture device, and (3) the calibration parameter, the depth value corresponding to the distance to the object based on the first image and the second image.

Assignees

• Intel Corp

Inventors

• Zabatani Aviad
• Bareket Sagy
• Menashe Ohad
• Sperling Erez
• Bronstein Alex
• Bronstein Michael
• Kimmel Ron
• Surazhsky Vitaly

Classifications

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• H04N13/243

  using three or more two-dimensional [2D] image sensors · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• H04N13/25

  using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor · CPC title

• H04N13/246 Primary

  Calibration of cameras · CPC title