Systems, methods, and media for stochastic exposure coding that mitigates multi-camera interference in continuous wave time-of-flight imaging

Patent metadata

Field	Value
Publication number	US-11474249-B2
Application number	US-201916556119-A
Country	US
Kind code	B2
Filing date	Aug 29, 2019
Priority date	Aug 29, 2019
Publication date	Oct 18, 2022
Grant date	Oct 18, 2022

Abstract

In accordance with some embodiments, systems, methods and media for stochastic exposure coding for continuous time-of-flight imaging are provided. In some embodiments, a method for estimating the depth of a scene is provided, comprising: stochastically selecting active slots based on a probability p; causing, during active slots, a light source to emit light modulated by a first modulation function toward a scene; causing, during active slots, an image sensor to generate a first, second, and third value based on received light from a portion of the scene and a first, second, and third demodulation function, respectively; inhibiting the light source during inactive slots; determining, for each of the active slots, depth estimates for the portion of the scene based on the first, second, and third value; and determining a depth estimate for the portion of the scene based on the depth estimates for the active slots.

First claim

What is claimed is:

1. A system for estimating the depth of a scene, the system comprising:
a light source;
an image sensor comprising at least one pixel;
a signal generator configured to output at least:
a first signal corresponding to a modulation function;

at least one hardware processor that is programmed to:
stochastically select, from a plurality of slots each corresponding to a portion of a total capture time, a first subset of the plurality of slots as active slots and a second subset of the plurality of slots as inactive slots based on a probability p of activating each slot of the plurality of slots;
cause, during each of the active slots, the light source to emit first modulated light toward the scene with modulation based on the first signal;
cause, during each of the active slots, the image sensor to generate a first value based on the light received from a portion of the scene and a second signal corresponding to a first demodulation function;
cause, during each of the active slots, the image sensor to generate a second value based on light received from the portion of the scene and a third signal corresponding to a second demodulation function;
cause, during each of the active slots, the image sensor to generate a third value based on light received from the portion of the scene and a fourth signal corresponding to a third demodulation function;
inhibit, during each of the inactive slots, the light source from emitting modulated light toward the scene;
determine, for each of a plurality of the active slots, a depth estimate for the portion of the scene based on the first value, the second value, and the third value; and
determine, for the total capture time, a depth estimate for the portion of the scene based on the depth estimates for each of the plurality of the active slots.

2. The system of claim 1 , the at least one hardware processor that is further programmed to:
determine, for each active slot, a total intensity received by the image sensor during the active slot;
determine that a clash has occurred for each active slot in which the total intensity exceeds a threshold; and
in response to determining that a clash has occurred for a particular active slot, exclude that active slot from the plurality of active slots.

3. The system of claim 2 , wherein the at least one hardware processor is further programmed to:
determine a mean total intensity value based on the total intensity values for each active slot; and
set the threshold based on a standard deviation of the total intensity values and the mean total intensity value.

4. The system of claim 1 , wherein each slot has a duration of in a range of 0.5 ms to 1.0 ms.

5. The system of claim 1 , wherein the at least one hardware processor that is further programmed to:
determine, for all of the active slots, a depth estimate for the portion of the scene based on the first value, the second value, and the third value; and
determine, for the total capture time, a depth estimate for the portion of the scene based on the depth estimates for each of the active slots.

6. The system of claim 1 , wherein the probability p of activating each slot of the plurality of slots is based on the relationship

p
=

1

A
0

,

where A 0 represents a peak power of the light source.

7. A method for estimating the depth of a scene, the method comprising:
stochastically selecting, from a plurality of slots each corresponding to a portion of a total capture time, a first subset of the plurality of slots as active slots and a second subset of the plurality of slots as inactive slots based on a probability p of activating each slot of the plurality of slots;
causing, during each of the active slots, a light source to emit first modulated light toward the scene with modulation based on a first signal corresponding to a first modulation function;
causing, during each of the active slots, an image sensor to generate a first value based on the light received from a portion of the scene and a second signal corresponding to a first demodulation function;
causing, during each of the active slots, the image sensor to generate a second value based on light received from the portion of the scene and a third signal corresponding to a second demodulation function;
causing, during each of the active slots, the image sensor to generate a third value based on light received from the portion of the scene and a fourth signal corresponding to a third demodulation function;
inhibiting, during each of the inactive slots, the light source from emitting modulated light toward the scene;
determining, for each of a plurality of the active slots, a depth estimate for the portion of the scene based on the first value, the second value, and the third value; and
determining, for the total capture time, a depth estimate for the portion of the scene based on the depth estimates for each of the plurality of the active slots.

8. The method of claim 7 , further comprising:
determining, for each active slot, a total intensity received by the image sensor during the active slot;
determining that a clash has occurred for each active slot in which the total intensity exceeds a threshold; and
in response to determining that a clash has occurred for a particular active slot, excluding that active slot from the plurality of active slots.

9. The method of claim 8 , further comprising:
determining a mean total intensity value based on the total intensity values for each active slot; and
setting the threshold based on a standard deviation of the total intensity values and the mean total intensity value.

10. The method of claim 7 , wherein each slot has a duration of in a range of 0.5 ms to 1.0 ms.

11. The method of claim 7 , further comprising:
determining, for all of the active slots, a depth estimate for the portion of the scene based on the first value, the second value, and the third value; and
determining, for the total capture time, a depth estimate for the portion of the scene based on the depth estimates for each of the active slots.

12. The method of claim 7 , wherein the probability p of activating each slot of the plurality of slots is based on the relationship

p
=

1

A
0

,

where A 0 represents a peak power of the light source.

13. A non-transitory computer readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for estimating the depth of a scene, the method comprising:
stochastically selecting, from a plurality of slots each corresponding to a portion of a total capture time, a first subset of the plurality of slots as active slots and a second subset of the plurality of slots as inactive slots based on a probability p of activating each slot of the plurality of slots;
causing, during each of the active slots, a light source to emit first modulated light toward the scene with modulation based on a first signal corresponding to a first modulation function;
causing, during each of the active sl

Assignees

• Wisconsin Alumni Res Found
• Wisconsin Alumni Reseach Found

Inventors

• Gupta Mohit
• Lee Jongho

Classifications

• G01S17/89 Primary

  for mapping or imaging · CPC title

• G01S7/493

  Extracting wanted echo signals · CPC title

• G01S17/10

  using transmission of interrupted, pulse-modulated waves (determination of distance by phase measurements G01S17/32) · CPC title

• G01S17/894 Primary

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• G06T7/521

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title