Devices and methods for an imaging system with a dual camera architecture

Patent metadata

Field	Value
Publication number	US-11611734-B2
Application number	US-202016830573-A
Country	US
Kind code	B2
Filing date	Mar 26, 2020
Priority date	Aug 7, 2013
Publication date	Mar 21, 2023
Grant date	Mar 21, 2023

Abstract

An electronic device can include a first image sensor configured to capture a first image of a field of view and a second image sensor configured to capture a second image of the field of view. The electronic device can include a color filter adjacent to the second image sensor such that the field of view is viewable by the second image sensor through the color filter. The first image can have a first pixel resolution. The second image can have a second pixel resolution. The electronic device can include a controller configured to determine a third image based on luminance content of the first image and color content of the second image. The third image can have a third pixel resolution indicative of a spatial resolution of the first image and a spectral resolution of the second image.

First claim

What is claimed is:

1. A device comprising:
a first camera that captures, via a first aperture at a first location of the device, a first image of a field of view, wherein the first image is a grayscale image;
a second camera that captures, via a second aperture at a second location of the device, a second image of the field of view, wherein the second camera comprises a color filter configured to selectively allow different wavelengths of light received via the second aperture to illuminate respective different sets of pixels of the second camera, wherein the second image is a color image including a representation of a color content of the field of view, and wherein the first location and second location are separated by a distance such that the contents of the first image and second image differ due to the parallax effect;
a display; and
a controller, coupled to the first camera and the second camera, wherein the controller is configured to:
determine a luminance content of the second image;
based on the luminance content of the second image being greater than a threshold, incorporate the color content obtained from the color image into the grayscale image obtained from the first camera to obtain a third image; and
cause the display to display the third image.

2. The device of claim 1 , further comprising:
a first lens that focuses light from the field of view into the first camera; and
a second lens that focuses light from the field of view into the second camera.

3. The device of claim 2 , further comprising:
a first image sensor included in the first camera, wherein the first image sensor receives at least a portion of the light focused by the first lens; and
a second image sensor included in the second camera, wherein the second image sensor receives at least a portion of the light focused by the second lens.

4. The device of claim 1 , wherein the color filter includes at least one of a Bayer filter, an RGBE filter, a CYYM filter, a CYGM filter, or an RGBW filter.

5. The device of claim 1 , wherein the greyscale image includes a representation of luminance content in the field of view, and wherein the third image includes luminance content of the first image and color content of the second image.

6. The device of claim 1 , wherein the first image has a first pixel resolution indicative of a spatial resolution of the first image, and wherein the second image has a second pixel resolution indicative of a spectral resolution of the second image.

7. The device of claim 6 , wherein the third image has a third pixel resolution indicative of the spatial resolution of the first image and the spectral resolution of the second image.

8. The device of claim 6 , wherein the first pixel resolution of the first image is greater than the second pixel resolution of the second image.

9. The device of claim 8 , wherein the third image has a third pixel resolution that is substantially same as the first pixel resolution of the first image.

10. The device of claim 1 , wherein the controller aligns features in the first image with corresponding features in the second image to obtain the third image.

11. A method comprising:
operating, by a computing device, a first camera to capture, via a first aperture at a first location, a first image of a field of view, wherein the first image is a grayscale image;
operating, by the computing device, a second camera, comprising a color filter, that captures, via a second aperture at a second location, a second image of the field of view, wherein the second image is a color image including a representation of a color content of the field of view at multiple different colors or wavelengths, and wherein the first location and second location are separated by a distance such that the contents of the first image and second image differ due to the parallax effect;
determining, by the computing device, a luminance content of the second image;
based on the luminance content of the second image being greater than a threshold, incorporating, by the computing device, the color content obtained from the color image into the grayscale image obtained from the first camera to obtain a third image; and
causing, by the computing device, a display to display the third image.

12. The method of claim 11 , wherein the color filter includes at least one of a Bayer filter, an RGBE filter, a CYYM filter, a CYGM filter, or an RGBW filter.

13. The method of claim 11 , wherein the grayscale image includes a representation of luminance content in the field of view, and wherein the third image includes luminance content of the first image and color content of the second image.

14. The method of claim 11 , wherein the first image has a first pixel resolution indicative of a spatial resolution of the first image, and wherein the second image has a second pixel resolution indicative of a spectral resolution of the second image.

15. The method of claim 14 , wherein the third image has a third pixel resolution indicative of the spatial resolution of the first image and the spectral resolution of the second image.

16. The method of claim 14 , wherein the first pixel resolution of the first image is greater than the second pixel resolution of the second image.

17. The method of claim 16 , wherein the third image has a third pixel resolution that is substantially same as the first pixel resolution of the first image.

18. The method of claim 11 , further comprising:
aligning, by the computing device, features in the first image with corresponding features in the second image to obtain the third image.

19. A non-transitory computer readable medium storing instructions that, when executed by one or more processors of a computing device, cause the computing device to perform operations comprising:
causing a first camera to capture, via a first aperture at a first location, a first image of a field of view, wherein the first image is a grayscale image;
causing a second camera that comprises a color filter to capture, via a second aperture at a second location, a second image of the field of view, wherein the second image is a color image including a representation of a color content of the field of view at multiple different colors or wavelengths, and wherein the first location and second location are separated by a distance such that the contents of the first image and second image differ due to the parallax effect;
determine a luminance content of the second image;
based on the luminance content of the second image being greater than a threshold, incorporating the color content obtained from the color image into the grayscale image obtained from the first camera to obtain a third image; and
causing a display to display the third image.

20. The non-transitory computer readable medium of claim 19 , wherein the operations further comprise:
aligning features in the first image with corresponding features in the second image to obtain the third image.

Assignees

• Google Llc

Inventors

• D'Amico Sam
• Miao Xiaoyu
• Parviz Babak

Classifications

• H04N23/45

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N5/265

  Mixing · CPC title

• H04N13/243

  using three or more two-dimensional [2D] image sensors · CPC title

• H04N13/239 Primary

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title