Three-dimensional imaging sensor calibration

Patent metadata

Field	Value
Publication number	US-2016360185-A1
Application number	US-201514730078-A
Country	US
Kind code	A1
Filing date	Jun 3, 2015
Priority date	Jun 3, 2015
Publication date	Dec 8, 2016
Grant date	—

Abstract

Technologies are generally described for calibrating three-dimensional image sensors. In some examples, an imaging system may include a sensor for detecting two-dimensional image data associated with a scene and a sensor for detecting depth data associated with the scene. Both sensors may also be configured to detect a reference signal used to illuminate the scene. The imaging system may then be configured to form three-dimensional data about the scene by using the reference signal to combine the two-dimensional image data and the depth data.

First claim

What is claimed is:

1 . A method to calibrate an image sensor, the method comprising:
detecting, at the image sensor, two-dimensional image data including a plurality of image pixels of a scene; detecting, at the image sensor, a reference signal associated with at least one image pixel of the plurality of image pixels, the reference signal transmitted from a vicinity of the image sensor to the scene; and determining, based on the detected reference signal, a depth associated with the at least one image pixel.

2 . The method of claim 1 , wherein detecting the reference signal comprises at least one of:
detecting an infrared laser signal; and detecting a structured light signal.

3 . The method of claim 1 , wherein determining the depth comprises determining the depth based on a time-of-flight parameter associated with the detected reference signal.

4 . The method of claim 1 , wherein:
determining the depth comprises mapping the at least one image pixel to at least one depth data element; and the at least one depth data element is associated with the detected reference signal.

5 . The method of claim 1 , wherein detecting the reference signal comprises detecting the reference signal using at least one of:
a first sensor pixel having an infrared filter and at least one color filter; and a second sensor pixel having an infrared filter and no color filters.

6 . The method of claim 1 , wherein:
detecting the two-dimensional image data comprises detecting the two-dimensional image data in at least one image frame of a sequence of image frames; and detecting the reference signal comprises detecting the reference signal in at least one other image frame of the sequence of image frames.

7 . The method of claim 6 , further comprising adjusting a frequency of detection of the reference signal in the at least one other image frame based on one or both of a frame rate associated with the sequence of image frames and scene dynamics.

8 . The method of claim 1 , wherein detecting the two-dimensional image data comprises:
determining that the detected reference signal is present in all image frames in a sequence of image frames; and retrieving data for an image pixel obscured by the detected reference signal in a first image frame from a corresponding image pixel in a second image frame.

9 . An image sensor system comprising:
an image sensor configured to detect two-dimensional image data associated with a scene, wherein the two-dimensional image data includes a plurality of image pixels; a reference signal filter configured to cause the image sensor to detect a returned reference signal associated with at least one image pixel of the plurality of image pixels, wherein the returned reference signal is transmitted from a vicinity of the image sensor onto the scene; and a processor configured to determine, based on the detected reference signal, a depth associated with the at least one image pixel.

10 . The system of claim 9 , wherein the reference signal comprises at least one of:
an infrared laser signal; and a structured light signal.

11 . The system of claim 9 , wherein the processor is further configured to determine the depth based on a time-of-flight parameter associated with the reference signal.

12 . The system of claim 9 , wherein the processor is configured to determine the depth by mapping the at least one image pixel to at least one depth data element associated with the detected reference signal.

13 . The system of claim 9 , wherein the reference signal filter includes at least one of:
an infrared filter for a first sensor pixel having at least one color filter; and an infrared filter for a second sensor pixel having no color filters.

14 . The system of claim 9 , wherein the image sensor is further configured to:
detect the two-dimensional image data in at least one image frame of a sequence of image frames; and detect the reference signal in at least one other image frame of the sequence of image frames.

15 . The system of claim 14 , wherein the processor is further configured to adjust a frequency of detection of the reference signal in the at least one other image frame based on one or both of a frame rate associated with the sequence of image frames and scene dynamics.

16 . The system of claim 9 , wherein the processor block is further configured to:
determine that the reference signal is present in all image frames in a sequence of image frames; and retrieve data for an image pixel in a first image frame obscured by the reference signal from a corresponding image pixel in a second image frame.

17 . An imaging system comprising:
a transmitter configured to transmit a reference signal; an image sensor configured to:
detect two-dimensional image data associated with a scene; and
detect the reference signal returned from the scene, wherein the reference signal is associated with at least one image pixel of a plurality of image pixels of the two-dimensional image data; and
a processor block configured to:
determine depth data based on the detected reference signal; and
form three-dimensional scene data based on the two-dimensional image data and the depth data.

18 . The system of claim 17 , wherein the transmitter is configured to transmit the reference signal as one of:
an infrared laser signal; and a structured light signal.

19 . The system of claim 17 , wherein the processor block is configured to determine the depth data based on a time-of-flight parameter associated with the reference signal.

20 . The system of claim 17 , wherein:
the processor block is configured to form the three-dimensional scene data by mapping at least one image pixel in the two-dimensional image data and at least one depth data element in the depth data using the reference signal.

21 . The system of claim 17 , wherein the image sensor includes at least one of:
an infrared filter for a first sensor pixel having at least one color filter; and an infrared filter for a second sensor pixel having no color filters.

22 . The system of claim 17 , wherein the image sensor is further configured to:
detect the two-dimensional image data in at least one image frame of a sequence of image frames; and detect the reference signal in at least one other image frame of the sequence of image frames.

23 . The system of claim 22 , wherein the processor block is further configured to adjust a frequency of detection of the reference signal in the at least one other image frame based on one or both of a frame rate associated with the sequence of image frames and scene dynamics.

24 . The system of claim 17 , wherein the processor block is further configured to:
determine that the reference signal is present in all image frames in a sequence of image frames; and retrieve data for an image pixel in a first image frame obscured by the reference signal from a corresponding image pixel in a second image frame.

Assignees

• Empire Technology Dev Llc

Inventors

• Margalit Mordehai
• Nehmadi Youval

Classifications

• H04N13/246 Primary

  Calibration of cameras · CPC title

• H04N13/254

  in combination with electromagnetic radiation sources for illuminating objects · CPC title

• H04N2013/0077

  Colour aspects · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title