Correction of depth images from T-O-F 3D camera with electronic-rolling-shutter for light modulation changes taking place during light integration

Patent metadata

Field	Value
Publication number	US-10061029-B2
Application number	US-201514883598-A
Country	US
Kind code	B2
Filing date	Oct 14, 2015
Priority date	Jan 6, 2015
Publication date	Aug 28, 2018
Grant date	Aug 28, 2018

Abstract

In embodiments, a T-O-F depth imaging device renders a depth image of an object that has corrected depth values. The device includes a pixel array that uses an Electronic Rolling Shutter scheme. The device also includes a light source that transmits towards the object light that is modulated at an operating frequency, and an optical shutter that opens and closes at the operating frequency. The optical shutter further modulates the light that is reflected from the object, before it reaches the pixel array. The transmission of the light and the operation of the shutter change phase relative to each other while at least one of the pixels is sensing the light it receives, which permits a faster frame rate. Depth is determined by the amount of light sensed by the pixels, and a correction is computed to compensate for the changing phase.

First claim

What is claimed is:

1. A T-O-F depth imaging device configured to render a depth image of an object, comprising:
a housing;
a controller configured to generate an illumination clock signal that is modulated at an operating frequency and a shutter clock signal that is modulated at the operating frequency, the shutter clock signal having a driver phase difference from the illumination clock signal;
a light source on the housing configured to emit light towards the object responsive to the illumination clock signal;
an optical shutter configured to alternately open and close responsive to the shutter clock signal;
a pixel array having a plurality of depth pixels arranged in a plurality of rows, a first one of the rows having at least a first depth pixel that senses, at least, a portion of the emitted light reflected from the object back towards the housing and further passes through the optical shutter while the optical shutter is open from a first time instant when the driver phase difference has a first angle value to a second time instant when the driver phase difference has a second angle value that is at least 25° different from the first angle value, the first depth pixel further configured to generate a first depth value based on the sensing of the portion of the emitted light; and
a processor configured to:
compute a first phase correction from an attribute of the first row, and
compute an adjusted first depth value from the first depth value and the first phase correction, and

in which a depth image of the object is rendered that includes the adjusted first depth value.

2. The T-O-F depth imaging device of claim 1 , in which
the sensing between the first time instant and the second time instant is continuous.

3. The T-O-F depth imaging device of claim 1 , in which
the attribute includes a location of the first row relative to locations of the plurality of the rows.

4. The T-O-F depth imaging device of claim 1 , in which
the adjusted first depth value is computed by subtracting the first phase correction from the first depth value.

5. The T-O-F depth imaging device of claim 1 , in which
the first row includes additional depth pixels that are configured to sense the portion of the emitted light concurrently with the first depth pixel and to generate additional respective depth values responsive to the sensing,
the processor is further configured to compute additional adjusted depth values from the additional depth values and the first phase correction, and
the rendered depth image includes the additional adjusted depth values.

6. The T-O-F depth imaging device of claim 5 , in which
the additional adjusted depth values are computed by subtracting the first phase correction from the additional depth values.

7. The T-O-F depth imaging device of claim 1 , in which
a second one of the rows different from the first row has at least a second depth pixel configured to sense the portion of the emitted light, the second depth pixel configured to thus sense from a third time instant when the driver phase difference has the first angle value to a fourth time instant when the driver phase difference has the second angle value, the second depth pixel further configured to generate a second depth value responsive to the sensing,
the processor is further configured to compute a second phase correction from an attribute of the second row, the second phase correction different from the first phase correction,
the processor is further configured to compute an adjusted second depth value from the second depth value and the second phase correction, and
the rendered depth image includes the adjusted second depth value.

8. The T-O-F depth imaging device of claim 7 , in which
the attribute of the second row includes a location of the second row relative to locations of the plurality of the rows.

9. The T-O-F depth imaging device of claim 7 , in which
the adjusted second depth value is computed by subtracting the second phase correction from the second depth value.

10. The T-O-F depth imaging device of claim 7 , in which
the second row includes additional depth pixels that are configured to sense the portion of the emitted light concurrently with the second depth pixel and to generate additional respective depth values responsive to the sensing,
the processor is further configured to compute additional adjusted depth values from the additional depth values and the second phase correction, and
the rendered depth image includes the additional adjusted depth values.

11. The T-O-F depth imaging device of claim 10 , in which
the additional adjusted depth values are computed by subtracting the second phase correction from the additional depth values.

12. A method for a T-O-F depth imaging device to render a depth image of an object, the T-O-F depth imaging device including a housing, a controller, a processor, a light source on the housing, an optical shutter, and a pixel array having a plurality of depth pixels arranged in a plurality of rows, a first one of the rows having at least a first depth pixel, the method comprising:
generating an illumination clock signal that is modulated at an operating frequency;
emitting, by the light source, light towards the object responsive to the illumination clock signal;
generating a shutter clock signal that is modulated at the operating frequency, the shutter clock signal having a driver phase difference from the illumination clock signal;
alternately opening and closing the optical shutter responsive to the shutter clock signal;
sensing, by the first depth pixel, a portion of the emitted light that is reflected from the object back towards the housing and further passes through the optical shutter while the optical shutter is open, the first depth pixel thus sensing from a first time instant when the driver phase difference has a first angle value to a second time instant when the driver phase difference has a second angle value that is at least 25° different from the first angle value,
generating, by the first depth pixel, a first depth value responsive to the sensing;
computing a first phase correction from an attribute of the first row;
computing an adjusted first depth value from the first depth value and the first phase correction; and
rendering a depth image of the object that includes the adjusted first depth value.

13. The method of claim 12 , in which
the sensing between the first time instant and the second time instant is continuous.

14. The method of claim 12 , in which
the attribute includes a location of the first row relative to locations of the plurality of the rows.

15. The method of claim 12 , in which
the adjusted first depth value is computed by subtracting the first phase correction from the first depth value.

16. The method of claim 12 , in which
the first row includes additional depth pixels that are configured to sense the portion of the emitted light concurrently with the first depth pixel and to generate additional respective depth values responsive to the sensing,
the processor is configured to compute additional adjusted depth values from the additional depth values and the first phase correction, and
the rendered depth image includes the additional adjusted depth values.

17. The method of claim 16 , in which
the additional adjusted depth values are computed by subtracting the first phase correction from the additional depth values.

18. The method of claim 12 , in which
a second one of the rows different from the first row has at least a seco

Assignees

• Waligorski Gregory
• Ovsiannikov Ilia
• Park Yong Hwa
• Samsung Electronics Co Ltd

Inventors

• Waligorski Gregory
• Ovsiannikov Ilia
• Park Yong-Hwa

Classifications

• G01S7/4914

  of detector arrays, e.g. charge-transfer gates · CPC title

• G01S17/894

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• G01S17/36

  with phase comparison between the received signal and the contemporaneously transmitted signal · CPC title

• G01S7/4816 Primary

  of receivers alone · CPC title

• G01S17/89 Primary

  for mapping or imaging · CPC title