Dual-band stereo depth sensing system

Patent metadata

Field	Value
Publication number	US-10628952-B2
Application number	US-201715837961-A
Country	US
Kind code	B2
Filing date	Dec 11, 2017
Priority date	Dec 11, 2017
Publication date	Apr 21, 2020
Grant date	Apr 21, 2020

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, that determine whether to illuminate a scene with electromagnetic radiation at a first wavelength or electromagnetic radiation at a second wavelength. An illuminator that is configured to selectively emit electromagnetic radiation at the first wavelength and at the second wavelength is triggered to emit electromagnetic radiation at the first wavelength instead of the second wavelength. A first voltage is applied to a first electrically active filter. A third voltage is applied to a second electrically active filter. A first image is obtained from a first sensor that senses electromagnetic radiation that passes through the first filter. A second image is obtained from a second sensor that senses electromagnetic radiation that passes through the second filter. A depth map is generated from the first image and the second image.

First claim

What is claimed is:

1. A device comprising:
a first sensor including a first electrically active filter that is configured to:
(i) when a first voltage is applied to the first electrically active filter, pass through electromagnetic radiation at a first wavelength and attenuate electromagnetic radiation at a second wavelength, and
(ii) when a second voltage is applied to the first electrically active filter, attenuate electromagnetic radiation at the first wavelength and pass through electromagnetic radiation at the second wavelength;
a second sensor including a second electrically active filter that is configured to:
(i) when a third voltage is applied to the second electrically active filter, pass through electromagnetic radiation at the first wavelength and attenuate electromagnetic radiation at the second wavelength, and
(ii) when a fourth voltage is applied to the second electrically active filter, attenuate electromagnetic radiation at the first wavelength and pass through electromagnetic radiation at the second wavelength;
an illuminator configured to selectively emit electromagnetic radiation at the first wavelength and at the second wavelength; and
electronic circuitry configured to:
in response to receiving an instruction to illuminate a scene and sense the scene, determine, based on environmental light conditions, whether to illuminate the scene and sense the scene with electromagnetic radiation with electromagnetic radiation at the first wavelength or electromagnetic radiation at the second wavelength,
in response to determining to illuminate the scene and sense the scene with electromagnetic radiation at the first wavelength:
(a) trigger the illuminator to emit electromagnetic radiation at the first wavelength,
(b) apply the first voltage to the first electrically active filter, and
(c) apply the third voltage to the second electrically active filter; and
in response to determining to illuminate the scene and sense the scene with electromagnetic radiation at the second wavelength:
(a) trigger the illuminator to emit electromagnetic radiation at the second wavelength,
(b) apply the second voltage to the first electrically active filter, and
(c) apply the fourth voltage to the second electrically active filter.

2. The device of claim 1 , wherein the electronic circuitry is configured to:
obtain a first image from the first sensor;
obtain a second image from the second sensor; and
generate a depth map from the first image and the second image.

3. The device of claim 2 , wherein the electronic circuitry is configured to:
obtain the first image from the first sensor simultaneously with obtaining the second image from the second sensor; and
trigger the illuminator to emit electromagnetic radiation at the first wavelength in synchrony with obtaining the first image and obtaining the second image.

4. The device of claim 1 , wherein the determining to illuminate the scene and sense the scene with electromagnetic radiation at the first wavelength, comprises:
determining that electromagnetic radiation at the second wavelength sensed by the first sensor satisfies a criterion.

5. The device of claim 4 , wherein determining that electromagnetic radiation at the second wavelength sensed by the first sensor satisfies a criterion, comprises:
determining that an amount of electromagnetic radiation at the second wavelength sensed by the first sensor is more than an amount of electromagnetic radiation at the first wavelength sensed by the first sensor.

6. The device of claim 1 , wherein determining to illuminate the scene and sense the scene with electromagnetic radiation at the first wavelength, comprises:
determining that the device is currently located outdoors.

7. The device of claim 1 , wherein the illuminator comprises a first emitter that is configured to emit electromagnetic radiation at the first wavelength and a second emitter that is configured to emit electromagnetic radiation at the second wavelength.

8. The device of claim 1 , wherein the first sensor comprises a sensing element that is configured to sense electromagnetic radiation at both the first wavelength and the second wavelength.

9. The device of claim 8 , wherein the sensing element comprises a rolling shutter sensing element.

10. The device of claim 8 , wherein the first sensor comprises a notch filter between the sensing element and the first electrically active filter, wherein the notch filter is configured to attenuate electromagnetic radiation at wavelengths between the first wavelength and the second wavelength.

11. The device of claim 1 , wherein the first voltage and the third voltage are the same and the second voltage and the fourth voltage are the same.

12. A computer-implemented method, comprising:
determining, based on environmental light conditions, whether to illuminate a scene with electromagnetic radiation at a first wavelength or electromagnetic radiation at a second wavelength;
in response to determining to illuminate the scene with electromagnetic radiation at the first wavelength:
triggering an illuminator that is configured to selectively emit electromagnetic radiation at the first wavelength and at the second wavelength to emit electromagnetic radiation at the first wavelength instead of the second wavelength;
applying a first voltage to a first electrically active filter that is configured (i) to pass through electromagnetic radiation at the first wavelength and attenuate electromagnetic radiation at the second wavelength when the first voltage is applied to the first electrically active filter, and (ii) to attenuate electromagnetic radiation at the first wavelength and pass through electromagnetic radiation at the second wavelength when a second voltage is applied to the first electrically active filter;
applying a third voltage to a second electrically active filter that is configured (i) to pass through electromagnetic radiation at the first wavelength and attenuate electromagnetic radiation at the second wavelength when a third voltage is applied to the second electrically active filter, and (ii) to attenuate electromagnetic radiation at the first wavelength and pass through electromagnetic radiation at the second wavelength when a fourth voltage is applied to the second electrically active filter;
obtain a first image from a first sensor that senses electromagnetic radiation that passes through the first filter;
obtain a second image from a second sensor that senses electromagnetic radiation that passes through the second filter; and
generate a depth map from the first image and the second image

determining, based on the environmental light conditions, whether to illuminate the scene with electromagnetic radiation at the first wavelength or electromagnetic radiation at the second wavelength;
in response to determining to illuminate the scene with electromagnetic radiation at the second wavelength:
triggering an illuminator that is configured to selectively emit electromagnetic radiation at the first wavelength and at the second wavelength to emit electromagnetic radiation at the second wavelength instead of the first wavelength;
applying the second voltage to the first electrically active filter;
applying the fourth voltage to the second electrically active filter;
obtain a third image from the first sensor that senses electromagnetic radiation that passes through the first filter;
obtain a fourth image from the second sensor that senses electromagnetic radiation that passes through the second filter; and
generate a second depth map from the third image and the fourth image.

13. The method of claim 12 ,

Assignees

• Google Llc

Inventors

• Wan Chung Chun

Classifications

• H04N13/254

  in combination with electromagnetic radiation sources for illuminating objects · CPC title

• H04N13/271

  wherein the generated image signals comprise depth maps or disparity maps · CPC title

• H04N13/239

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title

• G03B15/05

  Combinations of cameras with electronic flash apparatus; Electronic flash units · CPC title

• G02F1/25 Primary

  as to hue or predominant wavelength · CPC title