Time-of-flight (TOF) assisted structured light imaging

Patent metadata

Field	Value
Publication number	US-10061028-B2
Application number	US-201414478858-A
Country	US
Kind code	B2
Filing date	Sep 5, 2014
Priority date	Sep 5, 2013
Publication date	Aug 28, 2018
Grant date	Aug 28, 2018

Abstract

A method for computing a depth map of a scene in a structured light imaging system including a time-of-flight (TOF) sensor and a projector is provided that includes capturing a plurality of high frequency phase-shifted structured light images of the scene using a camera in the structured light imaging system, generating, concurrently with the capturing of the plurality of high frequency phase-shifted structured light images, a time-of-flight (TOF) depth image of the scene using the TOF sensor, and computing the depth map from the plurality of high frequency phase-shifted structured light images wherein the TOF depth image is used for phase unwrapping.

First claim

What is claimed is:

1. A method for computing a depth map of a scene in a structured light imaging system comprising a time-of-flight (TOF) sensor and a projector configured to project a plurality of phase-shifted structured light patterns, the method comprising:
capturing a plurality of high frequency phase-shifted structured light images of the scene using a camera in the structured light imaging system;
generating, concurrently with the capturing of the plurality of high frequency phase-shifted structured light images, a time-of-flight (TOF) depth image of the scene using the TOF sensor; and
computing the depth map from the plurality of high frequency phase-shifted structured light images by:
mapping a location in a plane of the projector to a first location in an image of the camera using the TOF depth image, wherein the TOF depth image is used for phase unwrapping;
searching a neighborhood of locations in the plurality of high frequency phase-shifted structured light images around the first location to find a second location corresponding to the projector plane location, wherein the second location corresponds to the projector plane location when a phase value of the second location is equal to a phase value of the projector plane location; and
computing a depth value for the projector location based on the second location.

2. The method of claim 1 , wherein mapping a location comprises:
mapping the projector plane location to a location in the TOF depth image using a projector-TOF sensor calibration matrix; and
mapping the location in the TOF depth image to the first location using a camera-TOF sensor calibration matrix.

3. The method of claim 1 , wherein the camera is the TOF sensor.

4. The method of claim 3 , wherein the projector is a digital micromirror device using a phase modulated light emitting diode as a light source.

5. A structured light imaging system comprising:
a projector configured to project a plurality of phase-shifted structured light patterns; and
a time-of-flight (TOF) sensor,
wherein the structured light imaging system is configured to compute a depth map of a scene by performing a method comprising:
capturing a plurality of high frequency phase-shifted structured light images of the scene using a camera in the structured light imaging system;
generating, concurrently with the capturing of the plurality of high frequency phase-shifted structured light images, a time-of-flight (TOF) depth image of the scene using the TOF sensor; and
computing the depth map from the plurality of high frequency phase-shifted structured light images by:
mapping a location in a plane of the projector to a first location in an image of the camera using the TOF depth image, wherein the TOF depth image is used for phase unwrapping;
searching a neighborhood of locations in the plurality of high frequency phase-shifted structured light images around the first location to find a second location corresponding to the projector plane location, wherein the second location corresponds to the projector plane location when a phase value of the second location is equal to a phase value of the projector plane location; and
computing a depth value for the projector location based on the second location.

6. The structured light imaging system of claim 5 , wherein mapping a location comprises:
mapping the projector plane location to a location in the TOF depth image using a projector-TOF sensor calibration matrix; and
mapping the location in the TOF depth image to the first location using a camera-TOF sensor calibration matrix.

7. The structured light imaging system of claim 5 , wherein the camera is the TOF sensor.

8. The structured light imaging system of claim 7 , wherein the projector is a digital micromirror device using a phase modulated light emitting diode as a light source.

9. A non-transitory computer readable medium storing software instructions that, when executed by at least one processor in a structured light imaging system comprising a time-of-flight (TOF) sensor and a projector configured to project a plurality of phase-shifted structured light patterns, cause execution of a method for computing a depth map of a scene, the method comprising:
capturing a plurality of high frequency phase-shifted structured light images of the scene using a camera in the structured light imaging system;
generating, concurrently with the capturing of the plurality of high frequency phase-shifted structured light images, a time-of-flight (TOF) depth image of the scene using the TOF sensor; and
computing the depth map from the plurality of high frequency phase-shifted structured light images by:
mapping a location in a plane of the projector to a first location in an image of the camera using the TOF depth image, wherein the TOF depth image is used for phase unwrapping;
searching a neighborhood of locations in the plurality of high frequency phase-shifted structured light images around the first location to find a second location corresponding to the projector plane location, wherein the second location corresponds to the projector plane location when a phase value of the second location is equal to a phase value of the projector plane location; and
computing a depth value for the projector location based on the second location.

10. The non-transitory computer readable medium of claim 9 , wherein mapping a location comprises:
mapping the projector plane location to a location in the TOF depth image using a projector-TOF sensor calibration matrix; and
mapping the location in the TOF depth image to the first location using a camera-TOF sensor calibration matrix.

11. The non-transitory computer readable medium of claim 9 , wherein the camera is the TOF sensor.

12. The non-transitory computer readable medium of claim 11 , wherein the projector is a digital micromirror device using a phase modulated light emitting diode as a light source.

Assignees

• Texas Instruments Inc

Inventors

• Koppal Sanjeev Jagannatha
• Appia Vikram Vijayanbabu

Classifications

• G01B11/2527 Primary

  with phase change by in-plane movement of the patern · CPC title

• G01S17/46

  Indirect determination of position data · CPC title

• G01S17/894

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• G01S17/89 Primary

  for mapping or imaging · CPC title