Systems, methods, and media for encoding and decoding signals used in time of flight imaging

Patent metadata

Field	Value
Publication number	US-10739447-B2
Application number	US-201715699623-A
Country	US
Kind code	B2
Filing date	Sep 8, 2017
Priority date	Apr 20, 2017
Publication date	Aug 11, 2020
Grant date	Aug 11, 2020

Abstract

In accordance with some embodiments, systems, methods and media for encoding and decoding signals used in time-of-flight imaging are provided. In some embodiments, a method for estimating the depth of a scene is provided, comprising: causing a light source to emit modulated light toward the scene based on a modulation function; causing the image sensor to generate a first value based on the modulated light and a first demodulation function of K modulation functions; causing the image sensor to generate a second value; causing the image sensor to generate a third value; and determining a depth estimate for the portion of the scene based on the first value, the second value, the third value, and three correlation functions each including at least one half of a trapezoid wave.

First claim

What is claimed is:

1. A system for estimating the depth of a scene, the system comprising:
a light source;
an image sensor comprising at least one pixel;
a signal generator configured to output at least:
a first signal corresponding to a modulation function, wherein the modulation function comprises a square wave; and
a second signal corresponding to a first demodulation function of K demodulation functions; and

at least one hardware processor that is programmed to:
cause the light source to emit first modulated light toward the scene with modulation based on the first signal;
cause the image sensor to generate a first value based on the first modulated light received from a portion of the scene and the second signal;
cause the image sensor to generate a second value based on second modulated light received from the portion of the scene, and a third signal corresponding to a second demodulation function of the K demodulation functions;
cause the image sensor to generate a third value based on third modulated light received from the portion of the scene and a fourth signal corresponding to a third demodulation function of the K demodulation functions; and
determine a depth estimate for the portion of the scene based on the first value, the second value, the third value, and at least three correlation functions each including at least one half of a trapezoid wave.

2. The system of claim 1 , wherein K is at least four.

3. The system of claim 1 , wherein the at least one hardware processor is further programmed to cause the light source to emit the first modulated light during a first time period, the second modulated light during a second time period, and the third modulated light during a third time period, wherein the first value is based on the modulated light emitted during the first time period, the second value is based on the modulated light emitted during the second time period, and the third value is based on the modulated light emitted during the third time period.

4. The system of claim 3 , wherein the at least one hardware processor is further programmed to cause the light source to emit the first modulated light after the second time period has elapsed.

5. The system of claim 1 , wherein the modulation function has a duty cycle of one sixth of an integration period τ.

6. The system of claim 1 , wherein each of the K demodulation functions includes at least one square wave, and the K demodulation functions correspond to K correlation functions that each include at least one half of a trapezoid wave.

7. The system of claim 1 , wherein the signal generator comprises multiple signal generators.

8. A method for estimating the depth of a scene, the method comprising:
causing a light source to emit first modulated light toward the scene with modulation based on a first signal output by a signal generator, wherein the first signal corresponds to a modulation function comprising a square wave;
causing an image sensor to generate a first value based on the first modulated light received from a portion of the scene and a second signal output by the signal generator, wherein the second signal corresponds to a first demodulation function of K demodulation functions;
causing the image sensor to generate a second value based on second modulated light received from the portion of the scene and a third signal corresponding to a second demodulation function of the K demodulation functions;
causing the image sensor to generate a third value based on the third modulated light received from the portion of the scene and a fourth signal corresponding to a third demodulation function of the K demodulation functions; and
determining a depth estimate for the portion of the scene based on the first value, the second value, the third value, and at least three correlation functions each including at least one half of a trapezoid wave.

9. The method of claim 8 , wherein K is at least four.

10. The method of claim 8 , wherein causing a light source to emit modulated light toward the scene comprises:
causing the light source to emit the first modulated light during a first time period;
causing the light source to emit the second modulated light during a second time period; and
causing the light source to emit the third modulated light during a third time period, wherein the first value is based on the modulated light emitted during the first time period, the second value is based on the modulated light emitted during the second time period, and the third value is based on the modulated light emitted during the third time period.

11. The method of claim 10 , further comprising causing the light source to emit the first modulated light after the second time period has elapsed.

12. The method of claim 9 , wherein the modulation function has a duty cycle of one sixth of an integration period τ.

13. The method of claim 9 , wherein each of the K demodulation functions includes at least one square wave, and the K demodulation functions correspond to K correlation functions that each include at least one half of a trapezoid wave.

14. The method of claim 8 , wherein signal generator comprises multiple signal generators.

15. A non-transitory computer readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for estimating the depth of a scene, the method comprising:
causing a light source to emit first modulated light toward the scene with modulation based on a first signal output by a signal generator, wherein the first signal corresponds to a modulation function comprising a square wave;
causing an image sensor to generate a first value based on the modulated light received from a portion of the scene and a second signal output by the signal generator, wherein the second signal corresponds to a first demodulation function of K demodulation functions;
causing the image sensor to generate a second value based on second modulated light received from the portion of the scene and a third signal corresponding to a second demodulation function of the K demodulation functions;
causing the image sensor to generate a third value based on third modulated light received from the portion of the scene and a fourth signal corresponding to a third demodulation function of the K demodulation functions; and
determining a depth estimate for the portion of the scene based on the first value, the second value, the third value, and at least three correlation functions each including at least one half of a trapezoid wave.

16. The non-transitory computer readable medium of claim 15 , wherein K is at least four.

17. The non-transitory computer readable medium of claim 15 , wherein causing a light source to emit modulated light toward the scene comprises:
causing the light source to emit the modulated light during a first time period;
causing the light source to emit the modulated light during a second time period; and
causing the light source to emit the first modulated light during a third time period, wherein the first value is based on the second modulated light emitted during the first time period, the second value is based on the third modulated light emitted during the second time period, and the third value is based on the modulated light emitted during the third time period.

18. The non-transitory computer readable medium of claim 17 , wherein the method further comprises causing the light source to emit the first modulated light after the second time period has elapsed.

19.

Assignees

• Wisconsin Alumni Res Found
• Univ Columbia

Inventors

• Gutierrez Barragan Felipe
• Gupta Mohit
• Velten Andreas
• Breitbach Eric
• Nayar Shree K

Classifications

• H04N23/60

  Control of cameras or camera modules · CPC title

• H04N23/56

  provided with illuminating means · CPC title

• G01S17/894

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• G01S17/36

  with phase comparison between the received signal and the contemporaneously transmitted signal · CPC title

• G01S7/4915

  Time delay measurement, e.g. operational details for pixel components (signal extraction and conditioning G01S7/493); Phase measurement · CPC title