Time of flight sensor binning

Patent metadata

Field	Value
Publication number	US-9134114-B2
Application number	US-201313792431-A
Country	US
Kind code	B2
Filing date	Mar 11, 2013
Priority date	Mar 11, 2013
Publication date	Sep 15, 2015
Grant date	Sep 15, 2015

Abstract

A time-of-flight sensor device generates and analyzes a high-resolution depth map frame from a high-resolution image to determine a mode of operation for the time-of-flight sensor and an illuminator and to control the time-of-flight sensor and illuminator according to the mode of operation. A binned depth map frame can be created from a binned image from the time-of-flight sensor and combined with the high-resolution depth map frame to create a compensated depth map frame.

First claim

What is claimed is:

1. A method for use with a time-of-flight sensor, the method comprising:
generating by at least one processing device at least one high-resolution depth map frame from at least one high-resolution image received from the time-of-flight sensor;
analyzing by the at least one processing device at least one of the at least one high-resolution depth map frame and the at least one high-resolution image;
determining by the at least one processing device a mode of operation having an illuminator power output level and a binning factor for at least one binned lower-resolution image;
generating at the at least one processing device at least one binned lower-resolution depth map frame from the at least one binned lower-resolution image received from the time-of-flight sensor;
combining information from the at least one high-resolution depth map frame with information from the at least one binned lower-resolution depth map frame to generate at least one compensated depth map frame; and
controlling by the at least one processing device the time-of-flight sensor and at least one illuminator according to the mode of operation.

2. The method of claim 1 , wherein the step of analyzing by the at least one processing device the at least one of the at least one high-resolution depth map frame and the at least one high-resolution image further comprises determining an ambient light interference level, wherein determining by the at least one processing device a mode of operation comprising an illuminator power output level further comprises:
determining whether the ambient light interference level is lower than a threshold ambient light interference level; and
determining a reduced illuminator power output level in response to determining that the ambient light interference level is lower than a threshold ambient light interference level;
and wherein the controlling by the at least one processing device the at least one illuminator according to the mode of operation further comprises reducing illuminator power output level according to the determined reduced illuminator power output level.

3. The method of claim 1 , wherein:
analyzing by the at least one processing device the at least one of the at least one high-resolution depth map frame and the at least one high-resolution image further comprises determining an ambient light interference level;
wherein determining by the at least one processing device a mode of operation comprising the binning factor for the at least one binned lower-resolution image further comprises:
determining whether the ambient light interference level is higher than a threshold ambient light interference level; and
determining an increased binning factor for the at least one binned lower-resolution image in response to determining that the ambient light interference level is higher than a threshold ambient light interference level;

and wherein the controlling by the at least one processing device the time-of-flight sensor according to the mode of operation further comprises capturing the at least one binned lower-resolution image according to the increased binning factor.

4. The method of claim 1 , wherein:
analyzing by the at least one processing device the at least one of the at least one high-resolution depth map frame and the at least one high-resolution image further comprises determining a depth of at least one foreground object;
determining by the at least one processing device a mode of operation comprising an illuminator power output level further comprises:
determining whether the depth of the at least one foreground object is lower than a threshold depth; and
determining a reduced illuminator power output level in response to determining that the depth of the at least one foreground object is lower than a threshold depth;

and wherein the controlling by the at least one processing device the at least one illuminator according to the mode of operation further comprises reducing illuminator power output level according to the determined reduced illuminator power output level.

5. The method of claim 1 , wherein:
analyzing by the at least one processing device the at least one of the at least one high-resolution depth map frame and the at least one high-resolution image further comprises determining a depth of at least one foreground object;
determining by the at least one processing device a mode of operation comprising the binning factor for the at least one binned lower-resolution image further comprises:
determining whether the depth of the at least one foreground object is lower than a threshold depth; and
determining an increased binning factor for the at least one binned lower-resolution image in response to determining that the depth of the at least one foreground object is lower than a threshold depth;

and wherein the controlling by the at least one processing device the time-of-flight sensor according to the mode of operation further comprises capturing the at least one binned lower-resolution image according to the increased binning factor.

6. The method of claim 1 , wherein the at least one high-resolution image comprises at least one binned image with a binning factor less-than or equal to the binning factor for the at least one binned lower-resolution image.

7. The method of claim 1 , wherein analyzing the at least one of the at least one high-resolution depth map frame and the at least one high-resolution image further comprises analyzing the at least one of the at least one high-resolution depth map frame and the at least one high-resolution image in combination with at least one of at least one previous image and at least one previous depth map frame.

8. An apparatus comprising:
at least one illuminator;
at least one time-of-flight image sensor;
at least one processing device operatively coupled to the at least one illuminator and the at least one time-of-flight image sensor, the at least one processing device configured to:
receive at least one high-resolution image from the at least one time-of-flight image sensor;
generate at least one high-resolution depth map frame using the at least one high-resolution image;
analyze using a scene analysis module at least one of the at least one high-resolution image and the at least one high-resolution depth map frame;
determine using a binning mode configuration module and the analysis from the scene analysis module a mode of operation comprising an illuminator power output level and a binning factor for at least one binned lower-resolution image;
receive the at least one binned lower-resolution image from the at least one time-of-flight image sensor;
generate at least one binned lower-resolution depth map frame using the at least one binned lower-resolution image;
combine information from the at least one high-resolution depth map frame with information from the at least one binned lower-resolution depth map frame to generate at least one compensated depth map frame; and
control operation of the at least one time-of-flight image sensor and the at least one illuminator according to the mode of operation.

9. The apparatus of claim 8 , wherein the at least one processing device is further configured to:
analyze using the scene analysis module the at least one of the at least high-resolution depth map frame and the at least one high-resolution image to determine an ambient light interference level;
determine a mode of operation by:
determining whether the ambient light interference level is lower than a threshold ambient light interference level; and
determining a reduced illuminator power output level in response to determining that the ambient light interference level is lower than a threshold ambie

Assignees

• Texas Instruments Inc

Inventors

• Metz Werner Adam
• Ko Dong-Ik

Classifications

• G01B11/24 Primary

  for measuring contours or curvatures · CPC title

• G01S7/4918

  Controlling received signal intensity, gain or exposure of sensor · CPC title

• G01S7/4915

  Time delay measurement, e.g. operational details for pixel components (signal extraction and conditioning G01S7/493); Phase measurement · CPC title

• G01S17/08

  for measuring distance only (indirect measurement G01S17/46; active triangulation systems G01S17/48) · CPC title

• G01B11/026 Primary

  by measuring distance between sensor and object (G01B11/0608 takes precedence) · CPC title