Apparatus and method for generating depth image

Patent metadata

Field	Value
Publication number	US-9310488-B2
Application number	US-201113090683-A
Country	US
Kind code	B2
Filing date	Apr 20, 2011
Priority date	Sep 17, 2010
Publication date	Apr 12, 2016
Grant date	Apr 12, 2016

Abstract

An apparatus and method for generating a depth image are provided. The apparatus includes an optical modulator, an optical sensor, and a depth image processor. The optical modulator optically modulates light which is reflected from an object to which light modulated using a first pseudorandom (PN) sequence is irradiated. The optical modulator optically modulates the reflected light using second PN-sequences, where each of the second PN-sequences has a same PN sequence as the first PN-sequence, but the second PN-sequences have N different time shifts. The optical sensor senses the optically modulated reflective light. The depth image processor samples N intensity images from the optically modulated reflective light sensed by the optical sensor and generates a depth image of the object using the sampled N intensity images.

First claim

What is claimed is:

1. An apparatus for generating a depth image, the apparatus comprising:
an optical modulator which optically modulates reflective light which is reflected light from an object to which light modulated using a first pseudorandom (PN) sequence is irradiated, by use of second PN-sequences, each of which has a same PN sequence as the first PN-sequence, but the second PN-sequences having N different time shifts;
an optical sensor which senses the optically modulated reflective light; and
a depth image process unit which samples N intensity images from the optically modulated reflective light sensed by the optical sensor and generates a depth image of the object using the sampled N intensity images,
wherein the depth image process unit individually determines respective depth extraction errors of each of a plurality of depths when the plurality of depths are determined according to a plurality of times of flight of light that are predicted using the N sampled intensity images, and selects a depth of the plurality of depths having a smallest error deviation estimate of the errors as a final depth of the depth image,
wherein the depth image process unit calculates a time of flight of light that is irradiated from a light source and reaches the optical sensor based on the N sampled intensity images, and
wherein when N is a number that is equal to or greater than 3, the depth image process unit determines whether an intensity image I M havin a minimum intensity among the three or more sampled intensity images is a first captured intensity image I 1 or a thirdly captured intensity image I 3 , and calculates a depth according to the determination result.

2. The apparatus of claim 1 , wherein the first PN-sequence and the second PN-sequences are bipolar m-sequences, and
a minimum information unit transmission time T C and a length of sequence of the first PN-sequence are the same as a minimum information unit transmission time T C and a length of sequence, respectively, of the second PN-sequences.

3. The apparatus of claim 1 , wherein the first PN-sequence and the second PN-sequence are Walsh-codes or gold-sequences.

4. The apparatus of claim 1 , wherein the N intensity images that are sampled correspond respectively to the N different time shifts of the second PN-sequences.

5. The apparatus of claim 1 , wherein the depth image process unit models the sampled N intensity images using a function including the time of flight of light that is irradiated from the light source and reaches the optical sensor as an unknown quantity.

6. The apparatus of claim 5 , wherein the function including the time of flight as an unknown quantity further includes ambient light and reflectivity of the reflective light as unknown quantities.

7. The apparatus of claim 5 , wherein when four or more intensity images are sampled, the depth image process unit determines which intensity image I M has a minimum intensity and which two intensity images have two top intensities which are different from the minimum intensity among the four or more intensity images, and calculates a depth according to the determination result.

8. The apparatus of claim 5 , wherein the depth image process unit further determines the final depth by averaging the plurality of depths when the plurality of depths are determined according to the plurality of times of flight of light that are predicted using the N sampled intensity images.

9. The apparatus of claim 1 , further comprising:
the light source which emits light to an object;
a light source driver which drives the light source;
an optical modulator driver which drives the optical modulator; and
a control unit which controls the light source driver to generate a light driving waveform of the light source using the first PN-sequence, and controls the optical modulator driver to generate an optical modulation driving waveform of the optical modulator using the second PN-sequence.

10. The apparatus of claim 9 , wherein the depth image process unit increases a number of times of sampling the intensity images for expanding a capturing range of the depth image while the control unit maintains a constant interval between time shifts.

11. The apparatus of claim 9 , wherein the control unit increases precision of the depth image by controlling the light source driver and the optical modulator driver such that an interval between the time shifts is shorter than a minimum information unit transmission time T C of the first and second PN-sequences, and the depth image process unit samples the intensity images at the interval of the time shift.

12. The apparatus of claim 1 , wherein the first PN-sequence and the second PN-sequence are different from those of a different apparatus for generating a depth image.

13. The apparatus of claim 1 , wherein the first PN-sequence and the second PN-sequence are the same as those of a different apparatus for generating a depth image, but the first PN-sequence and the second PN-sequence are not synchronized with those of the different apparatus.

14. An apparatus for generating a depth image, the apparatus comprising:
an optical modulator which optically modulates reflective light which is reflected light from an object to which light modulated using N first pseudorandom (PN) sequences having, respectively, N different time shifts is irradiated, by use of a second PN-sequence which has a same PN sequence as that of the first PN-sequences but has a time shift different from those of the first PN-sequences;
an optical sensor which senses the optically modulated reflective light; and
a depth image process unit which samples N intensity images from light signals sensed by the optical sensor and generates a depth image of the object using the sampled N intensity images,
wherein the depth image process unit individually determines respective depth extraction errors of each of a plurality of depths when the plurality of depths are determined according to a plurality of times of flight of light that are predicted using the N sampled intensity images, and selects a depth of the plurality of depths having a smallest error deviation estimate of the errors as a final depth of the depth image,
wherein the depth image process unit calculates a time of flight of light that is irradiated from a light source and reaches the optical sensor based on the N sampled intensity images, and
wherein when N is a number that is equal to or greater than 3, the depth image process unit determines whether an intensity image I M havin a minimum intensity among the three or more sampled intensity images is a first captured intensity image I 1 or a thirdly captured intensity image I 3 , and calculates a depth according to the determination result.

15. The apparatus of claim 14 , wherein the depth image process unit models the sampled N intensity images with a function including the time of flight of light that is irradiated from the light source and reaches the optical sensor as an unknown quantity.

16. A method of generating a depth image, the method comprising:
performing optical modulation on reflective light which is reflected light from an object to which light modulated using a first pseudorandom (PN) sequence is irradiated, by use of a second PN-sequence which has a same PN sequence as a first PN-sequence but the second PN-sequence has N different time shifts;
sensing the optically modulated reflective light using an optical sensor;
sampling N intensity images obtained from the sensed optically modulated reflective light; and
generating a depth image of the object usin

Assignees

• Park Yong-Hwa
• Kim Eung-Sun
• Samsung Electronics Co Ltd

Inventors

• Park Yong-Hwa
• Kim Eung-Sun

Classifications

• H04N13/296

  Synchronisation thereof; Control thereof · CPC title

• H04N13/271 Primary

  wherein the generated image signals comprise depth maps or disparity maps · CPC title

• G01S17/894

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• H04N13/128

  Adjusting depth or disparity · CPC title

• G01S7/4816

  of receivers alone · CPC title