Depth sensor using modulated light projector and image sensor with color and IR sensing

Patent metadata

Field	Value
Publication number	US-9407837-B2
Application number	US-201414186877-A
Country	US
Kind code	B2
Filing date	Feb 21, 2014
Priority date	Feb 28, 2013
Publication date	Aug 2, 2016
Grant date	Aug 2, 2016

Abstract

An electronic device includes a modulated light projector to project an infrared light pattern and an image sensor. The image sensor includes an array of photosensors and a filter array overlying the array of photosensors. The filter array comprises a mosaic of color filter elements and infrared filter elements, each color filter element overlying a corresponding photosensor of a first subset of photosensors and each infrared filter element overlying a corresponding photosensor of a second subset of photosensors. The electronic device further includes a controller to, in a first mode, control the image sensor to capture a color image using the first subset of photosensors and, in a second mode, control the image sensor to capture a depth image representing a reflection of the infrared light pattern using the second subset of photosensors.

First claim

What is claimed is:

1. An electronic device comprising:
a modulated light projector to project an infrared light pattern; and
an image sensor comprising:
an array of photosensors; and
a filter array overlying the array of photosensors and comprising a mosaic of color filter elements and infrared filter elements, each color filter element overlying a corresponding photosensor of a first subset of photosensors and each infrared filter element overlying a corresponding photosensor of a second subset of photosensors, wherein each photosensor of the first subset has a first size and each photosensor of the second subset has a second size different than the first size; and
a controller to, in a first mode, control the image sensor to capture a color image using the first subset of photosensors and, in a second mode, control the image sensor to capture a depth image representing a reflection of the infrared light pattern using the second subset of photosensors.

2. The electronic device of claim 1 , wherein the modulated light projector comprises:
an array of one or more vertical cavity surface emitting laser (VCSEL) diodes;
an array of one or more lenses overlying the array of one or more VCSEL diodes; and
a diffractive optical element overlying the array of one or more lenses.

3. The electronic device of claim 1 , wherein the electronic device further comprises:
a display disposed at a first surface of the electronic device, wherein the electronic device is configured to present, via the display, imagery captured using the image sensor; and
wherein the modulated light projector and the image sensor are disposed at a second surface of the electronic device, the second surface opposite the first surface.

4. The electronic device of claim 3 , wherein:
the image sensor is a first image sensor; and
the electronic device further comprises a second image sensor disposed at the second surface, the second image sensor having a field of view different than a field of view of the first image sensor.

5. A method comprising:
providing an electronic device comprising an image sensor, the image sensor comprising an array of photosensors and a filter array overlying the array of photosensors and comprising a mosaic of color filter elements and infrared filter elements, each color filter element overlying a corresponding photosensor of a first subset of photosensors and each infrared filter element overlying a corresponding photosensor of a second subset of photosensors;
capturing, at the electronic device, a color image using the first subset of photosensors;
projecting, from the electronic device, an infrared light pattern;
capturing, at the electronic device, a depth image representing a reflection of the infrared light pattern using the second subset of photosensors; and
wherein providing the electronic device comprises providing the image sensor with each photosensor of the first subset having a first size and each photosensor of the second subset having a second size different than the first size.

6. The method of claim 5 , further comprising:
determining, at the electronic device, at least one spatial feature from at least one of the color image and the depth image; and
determining, at the electronic device, at least one of a relative position and a relative orientation of the electronic device based on the at least one spatial feature.

7. The method of claim 5 , further comprising:
displaying the color image at a display of the electronic device.

8. The method of claim 5 , wherein capturing the depth image comprises:
binning subarrays of M×N photosensors of the second subset into corresponding macropixels for the depth image, M and N comprising integer numbers greater than zero and at least one of M and N being greater than one.

9. The method of claim 5 , wherein providing the image sensor further comprises:
providing the image sensor with the color filter elements that are non-uniformly distributed in the filter array.

Assignees

• Google Inc

Inventors

• Lee Johnny C

Classifications

• G01C3/08 Primary

  Use of electric radiation detectors · CPC title

• H04N23/11

  for generating image signals from visible and infrared light wavelengths · CPC title

• H04N23/63

  by using electronic viewfinders · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• H04N23/73

  by influencing the exposure time · CPC title