Methods and apparatus for time-of-flight imaging

Patent metadata

Field	Value
Publication number	US-10191154-B2
Application number	US-201715431713-A
Country	US
Kind code	B2
Filing date	Feb 13, 2017
Priority date	Feb 11, 2016
Publication date	Jan 29, 2019
Grant date	Jan 29, 2019

Abstract

In some implementations, scene depth is extracted from dual frequency of a cross-correlation signal. A camera may illuminate a scene with amplitude-modulated light, sweeping the modulation frequency. For each modulation frequency in the sweep, each camera pixel may measure a cross-correlation of incident light and of a reference electrical signal. Each pixel may output a vector of cross-correlation measurements acquired by the pixel during a sweep. A computer may perform an FFT on this vector, identify a dual frequency at the second largest peak in the resulting power spectrum, and calculate scene depth as equal to a fraction, where the numerator is the speed of light times this dual frequency and the denominator is four times pi. In some cases, the two signals being cross-correlated have the same phase as each other during each cross-correlation measurement.

First claim

What is claimed is:

1. A method comprising:
(a) emitting amplitude-modulated light that illuminates a scene and varying modulation frequency of the light in a sweep of modulation frequency that includes three or more modulation frequencies;
(b) applying an electrical reference signal to a set of pixels in a camera and varying frequency of the reference signal in such a way that frequency of the reference signal is equal to modulation frequency of the light at each of the modulation frequencies of the sweep; and
(c) for each respective pixel in the set of pixels
(i) outputting a vector of cross-correlations measured by the respective pixel at different modulation frequencies during the sweep, in such a way that, for each given modulation frequency in the sweep, the vector includes a measured cross-correlation of the reference signal and of a signal comprising reflected light, which reflected light is at the given modulation frequency and is incident on the respective pixel after reflecting from the scene,
(ii) performing spectral analysis of a discrete-valued signal encoded by the vector, which spectral analysis computes a dual frequency of the discrete-valued signal, and
(iii) calculating, based on the dual frequency
(A) optical path length of a path along which light travels to the respective pixel, or
(B) depth of a scene point that corresponds to the respective pixel.

2. The method of claim 1 , wherein performing the spectral analysis includes performing a fast Fourier transform.

3. The method of claim 2 , wherein:
(a) the fast Fourier transform outputs a power spectrum, which power spectrum has at least two peaks; and
(b) the dual frequency occurs at the second highest peak in the power spectrum.

4. The method of claim 3 , wherein the calculating includes setting the depth equal to a fraction, where the numerator is speed of light times the dual frequency and the denominator is four times pi.

5. The method of claim 3 , wherein the calculating includes setting the optical path length equal to a fraction, where the numerator is speed of light times the dual frequency and the denominator is two times pi.

6. The method of claim 1 , wherein for each respective pixel in the set of pixels:
(a) the spectral analysis identifies a set of one or more dual frequencies of the vector;
(b) each respective dual frequency in the set of dual frequencies corresponds to a given path along which light travels to the respective pixel; and
(c) for each respective dual frequency in the set of dual frequencies, the calculating includes setting the optical path length of the given path equal to a fraction, where the numerator is speed of light times the respective dual frequency and the denominator is two times pi.

7. The method of claim 6 , wherein:
(a) the sweep has a bandwidth, which bandwidth is equal to the difference between the highest and lowest modulation frequencies in the sweep; and
(b) the method has an optical path length resolution such that a first optical path length and a second optical path length are resolvable by the method when the difference between the first and second optical path lengths is greater than a fraction, which fraction has a numerator equal to 1.206 times speed of light and a denominator equal to the bandwidth.

8. The method of claim 1 , wherein:
(a) the sweep has a bandwidth, which bandwidth is equal to the difference between the highest and lowest modulation frequencies in the sweep; and
(b) the method has a depth resolution such that a first depth and a second depth are resolvable by the method when the difference between the first and second depths is greater than a fraction, which fraction has a numerator equal to 0.603 times speed of light and a denominator equal to the bandwidth; and
(c) the first and second depths are each a depth of a scene point.

9. The method of claim 1 , wherein the optical path length or the depth are computed in accordance with the formula

ℱ
⁡

[

c
⁡

(

τ
,

f
M

)

]

⁢

(
κ
)

∝

δ
⁡

(
κ
)

+

δ
⁡

(

κ
±

2
⁢
π
⁢

⁢
z

c

)

,

where: (1) f M is modulation frequency of amplitude-modulated light; (2) c(τ, f M ) is cross-correlation of a first signal and a second signal, where the first signal is light incident on a pixel and the second signal is an electrical reference signal that is applied to the pixel, and where τ is lag between the first and second signals and where f M is temporal frequency of the first and second signals; (3) [⋅] is the discrete Fourier transform operator; (4) δ(⋅) is the Dirac delta function; (5) c is speed of light; (6) κ is dual frequency in a Fourier domain version of c(f M ); and (7) ∝ means “is proportional to”.

10. The method of claim 1 , wherein, at all times while cross-correlation for a given modulation frequency is being measured, the phase and temporal frequency of amplitude modulation of the emitted amplitude-modulated light is substantially the same as the phase and temporal frequency, respectively, of the reference signal.

11. The method of claim 1 , wherein, at all times while cross-correlation for a given modulation frequency is being measured, a phase difference is substantially constant, which phase difference is the difference between phase of amplitude modulation of emitted amplitude-modulated light and phase of the reference signal.

12. The method of claim 1 , wherein the light is incoherent before reaching the scene.

13. Apparatus comprising:
(a) means for emitting amplitude-modulated light that illuminates a scene, and for varying modulation frequency of the light in a sweep of modulation frequency that includes three or more modulation frequencies;
b) means for applying an electrical reference signal to a set of pixels in a camera;
(c) means for outputting, for each respective pixel in the set of pixels, a vect

Assignees

• Massachusetts Inst Technology

Inventors

• Kadambi Achuta
• Schiel James
• Bhandari Ayush
• Raskar Ramesh
• Taamazyan Vage

Classifications

• G01S7/4915 Primary

  Time delay measurement, e.g. operational details for pixel components (signal extraction and conditioning G01S7/493); Phase measurement · CPC title

• G01S17/10

  using transmission of interrupted, pulse-modulated waves (determination of distance by phase measurements G01S17/32) · CPC title

• G01S17/32

  using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated · CPC title

• G01S17/894

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• G01S17/34

  using transmission of continuous, frequency-modulated waves while heterodyning the received signal, or a signal derived therefrom, with a locally-generated signal related to the contemporaneously transmitted signal · CPC title