Real time calibration for time-of-flight depth measurement

Patent metadata

Field	Value
Publication number	US-11587245-B2
Application number	US-202016936345-A
Country	US
Kind code	B2
Filing date	Jul 22, 2020
Priority date	Sep 30, 2016
Publication date	Feb 21, 2023
Grant date	Feb 21, 2023

Abstract

A method for determining a distance to a target object includes transmitting light pulses to illuminate the target object and sensing, in a first region of a light-sensitive pixel array, light provided from an optical feedback device that receives a portion of the transmitted light pulses. The feedback optical device includes a preset reference depth. The method includes calibrating time-of-flight (TOF) depth measurement reference information based on the sensed light in the first region of the pixel array. The method further includes sensing, in a second region of the light-sensitive pixel array, light reflected from the target object from the transmitted light pulses. The distance of the target object is determined based on the sensed reflected light and the calibrated TOF measurement reference information.

First claim

What is claimed is:

1. A time-of-flight (TOF) imaging system, comprising:
an illuminator to transmit light pulses to illuminate a target object for determining a distance to the target object;
an image sensor having a light-sensitive pixel array to receive optical signals from the light pulses, the pixel array including an active region and a feedback region; and
an optical feedback device for directing a portion of the light from the illuminator to the feedback region of the pixel array, the optical feedback device including a preset reference depth;
wherein the imaging system is configured to:
in a calibration period,
transmit a group of calibration light pulses to illuminate the target object;
sense, in the feedback region of the pixel array, light from the optical feedback device, using a sequence of calibration shutter windows characterized by delay times representing a range of depth; and
calibrate TOF depth measurement reference information based on the sensed light in the feedback region of the pixel array;

in a measurement period,
transmit a first measurement light pulse to illuminate the target object;
sense, in the active region of the light-sensitive pixel array, light reflected from the target object, using a first measurement shutter window with a first measurement delay time relative to the first measurement light pulse;
transmit a second measurement light pulse to illuminate the target object;
sense, in the active region of the light-sensitive pixel array, light reflected from the target object, using a second measurement shutter window with a second measurement delay time relative to the second measurement light pulse; and
determine the distance of the target object based on light sensed using the first measurement shutter window and the second measurement shutter window and the calibrated TOF depth measurement reference information.

2. The system of claim 1 , wherein the sequence of calibration shutter windows comprises a first series of calibration windows and a second series of calibration windows, and delay times associated with the second series of calibration windows are offset from delay times associated with the first series of calibration windows by a width of the shutter window.

3. The system of claim 2 , wherein calibrating time-of-flight depth measurement reference information comprises associating the range of depth with ratios of light signals sensed using the first series of calibration windows and the second series of calibration windows.

4. The system of claim 2 , wherein the second measurement delay time is offset from the first measurement delay time by the width of the shutter window.

5. The system of claim 3 , wherein determining the distance of the target object comprises comparing a ratio of sensed light signals using the first measurement delay time and the second measurement delay time with the time-of-flight depth measurement reference information.

6. The system of claim 2 , wherein the optical feedback device comprises an optical fiber configured to couple light from inside an illuminator housing to the feedback region at a corner of the pixel array.

7. The system of claim 5 , wherein the TOF measurement reference information comprises a look-up table correlating a distance of an object to a ratio between two sampled light signals sensed using the first series of calibration windows and the second series of calibration windows.

8. The system of claim 1 , wherein the calibration period and the measurement period are included in a single frame period.

9. The system of claim 1 , wherein calibration takes about 1/300 of a frame period.

10. The system of claim 1 , further comprising multiple illumination sources and corresponding optical feedback regions.

11. The system of claim 1 , wherein the imaging system is configured to calibrate time-of-flight, TOF, depth measurement reference information after sensing, in the active region of the light-sensitive pixel array, light reflected from the target object.

12. A method for calibrating a time-of-flight (TOF) camera system, comprising:
transmitting a group of calibration light pulses to illuminate a target object;
sensing, in a first region of a light-sensitive pixel array, light provided from an optical feedback device that receives a portion of the group of calibration light pulses, using a sequence of calibration shutter windows characterized by delay times relative to the calibration light pulses representing a range of depth, the optical feedback device characterized by a preset reference depth;
calibrating TOF depth measurement information based on the light provided from the feedback optic device;
transmitting a first measurement light pulse to illuminate the target object;
sensing, in a second region of the light-sensitive pixel array, light reflected from the target object, using a first measurement shutter window with a first measurement delay time relative to the first measurement light pulse;
transmitting a second measurement light pulse to illuminate the target object;
sensing, in the second region of the light-sensitive pixel array, light reflected from the target object, using a second measurement shutter window with a second measurement delay time relative to the second measurement light pulse; and
determining a distance of the target object based on light sensed using the first measurement shutter window and the second measurement shutter window and the calibrated TOF depth measurement information.

13. The method of claim 12 , wherein the sequence of calibration shutter windows comprises a first series of calibration windows and a second series of calibration windows, and delay times associated with the second series of calibration windows are offset from delay times associated with the first series of calibration windows by a width of the shutter window.

14. The method of claim 13 , wherein calibrating TOF depth measurement reference information comprises associating the range of depth with ratios of light signals sensed using the first series of calibration windows and the second series of calibration windows.

15. The method of claim 14 , wherein the TOF depth measurement reference information comprises a look-up table correlating a distance of an object to a ratio between two sampled light signals using the first series of calibration windows and the second series of calibration windows.

16. In a digital camera characterized by a preset frame rate, a method, comprising:
in a single frame period determined by the preset frame rate,
transmitting calibration light pulses to illuminate a target object;
sensing, in a first region of a light-sensitive pixel array, light provided from an optical feedback device that receives a portion of the transmitted light pulses, the feedback optical device including a preset reference depth, wherein the light from the optical feedback device is sampled using a sequence of shutter windows that includes delay times relative to the transmitted calibration light pulses representing a range of distance;
calibrating time-of-flight (TOF) depth measurement reference information based on the sensed light in the first region of the pixel array;
transmitting measurement light pulses to illuminate a target object;
sensing, in a second region of the light-sensitive pixel array, light reflected from the target object from the transmitted light pulses using a measurement shutter windows with delay times relative to the measurement light pulses; and
determining a distance of the target object based on the sensed light reflected from the target obj

Assignees

• Magic Leap Inc

Inventors

• Cohen David
• Pellman Assaf
• Mazor Shai
• Tadmor Erez
• Yahav Giora

Classifications

• G01S17/08

  for measuring distance only (indirect measurement G01S17/46; active triangulation systems G01S17/48) · CPC title

• G01S7/497

  Means for monitoring or calibrating · CPC title

• G06T7/521 Primary

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title

• G06T2207/10004

  Still image; Photographic image · CPC title

• G01S7/4813 Primary

  Housing arrangements · CPC title