Multi-baseline camera array system architectures for depth augmentation in vr/ar applications

Patent metadata

Field	Value
Publication number	US-2016309134-A1
Application number	US-201615133186-A
Country	US
Kind code	A1
Filing date	Apr 19, 2016
Priority date	Apr 19, 2015
Publication date	Oct 20, 2016
Grant date	—

Abstract

Embodiments of the invention provide a camera array imaging architecture that computes depth maps for objects within a scene captured by the cameras, and use a near-field sub-array of cameras to compute depth to near-field objects and a far-field sub-array of cameras to compute depth to far-field objects. In particular, a baseline distance between cameras in the near-field subarray is less than a baseline distance between cameras in the far-field sub-array in order to increase the accuracy of the depth map. Some embodiments provide an illumination near-IR light source for use in computing depth maps.

First claim

What is claimed is:

1 . A method of estimating distances to objects within a scene from a set of images captured from different cameras in a set of cameras using a processor configured by an image processing application, the method comprising:
generating an initial depth map of a scene; identify near-field and far-field portions of the scene; refine the depth map for the near-field portions of the scene using image data captured from a near-field set of cameras; and refine the depth map for the far-filed portions of the scene using image data captured from a far-field set of cameras.

2 . The method of claim 1 , further comprising illuminating the scene using an illumination light source positioned within a threshold distance from a reference camera.

3 . The method of claim 1 , wherein the near-field portions of the scene are at a distance less than a certain threshold and the far-field portions of the scene are at a distance greater than the certain threshold.

4 . The method of claim 1 , wherein a baseline distance between cameras in the near-field set of cameras is less than a baseline distance between cameras in the far-filed set of cameras.

5 . The method of claim 1 , wherein each camera in the set of cameras has the same imaging properties.

6 . The method of claim 1 , where the set of cameras have different field of views.

7 . The method of claim 1 , wherein the set of cameras have different resolutions.

8 . The method of claim 1 , wherein the set of cameras image in different portions of a light spectrum.

9 . The method of claim 1 , wherein the set of cameras are monochrome cameras with a same field of view and resolution.

10 . The method of claim 9 , wherein the monochrome cameras image across at least a portion of the visible spectrum and at least a portion of the near-IR spectrum.

11 . The method of claim 1 , further comprising using the depth map in order to determine where to render an object on a display of a VR application such that the object is appropriately occluded by a real world object also being displayed within a VR application.

12 . The method of claim 1 , further comprising using the depth map in order to determine where to render an object on a display of an AR application such that the object is appropriately occluded by a real world object visible through a transparent display within the AR application.

13 . The method of claim 1 , wherein the set of cameras are mounted within a headset that includes a display via which images can be displayed.

14 . The method of claim 13 , wherein a reference camera and the display are part of a removable component that is mounted within the headset.

15 . A camera array comprising:
a first subarray of cameras positioned at a first location, the first camera array including at least two cameras; a second subarray of cameras positioned at a second location, the second camera array including at least two cameras; a third subarray of cameras positioned between the first location and the second locations, the third subarray of cameras including at least one reference camera; and an illumination light source positioned within a threshold distance from the reference camera.

16 . The camera array of claim 15 , wherein the cameras in the first, second, and third subarrays of cameras have the same imaging characteristics.

17 . The camera array of claim 15 , wherein the cameras in the first and second subarrays of cameras have different imaging characteristics from the at least one reference camera.

18 . The camera array of claim 15 , wherein the cameras in the first and second subarrays of cameras each have a different epipolar axis with the reference camera.

19 . The camera array of claim 15 , wherein the cameras are all monochrome cameras with a same field of view.

20 . The camera array of claim 19 , wherein the field of view is selected from the group consisting of 70 degrees and 90 degrees.

Assignees

• Pelican Imaging Corp

Inventors

• Venkataraman Kartik
• Lelescu Dan
• Duparre Jacques

Classifications

• H04N23/11

  for generating image signals from visible and infrared light wavelengths · CPC title

• H04N23/45

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N23/56

  provided with illuminating means · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G02B27/0172

  characterised by optical features · CPC title