Three-dimensional image sensors

Patent metadata

Field	Value
Publication number	US-2016309136-A1
Application number	US-201615193558-A
Country	US
Kind code	A1
Filing date	Jun 27, 2016
Priority date	Nov 8, 2012
Publication date	Oct 20, 2016
Grant date	—

Abstract

A single image sensor including an array of uniformly and continuously spaced light-sensing pixels in conjunction with a plurality of lenses that focus light reflected from an object onto a plurality of different pixel regions of the image sensor, each lens focusing light on a different one of the pixel regions enables a controller, including a processor and an object detection module, coupled to the single image to analyze the pixel regions, to generate a three-dimensional (3D) image of the object through a plurality of images obtained with the image sensor, generate a depth map that calculates depth values for pixels of at least the object, detect 3D motion of the object using the depth values, create a 3D model of the object based on the 3D image, and track 3D motion of the object based on the 3D model.

First claim

What is claimed is:

1 . A camera for capturing images of an object, the camera comprising:
a single image sensor including an array of uniformly and continuously spaced light-sensing pixels; a plurality of lenses that focus light reflected from an object onto a plurality of different pixel regions of the image sensor, each lens focusing light on a different one of the pixel regions; and a controller, including a processor and an object detection module, coupled to the single image sensor that:
analyzes the pixel regions to generate a three-dimensional (3D) image of the object through a plurality of images obtained with the image sensor;
generates a depth map that calculates depth values for pixels of at least the object;
detects 3D motion of the object using the depth values;
creates a 3D model of the object based on the 3D image; and
tracks 3D motion of the object based on the 3D model.

2 . The camera of claim 1 , wherein the controller is further configured to:
acquire additional images of the object as the object moves through space; analyze the additional images to generate additional three-dimensional images of the object; and update the 3D model to reflect 3D motion of the object based upon the additional three-dimensional (3D) images.

3 . The camera of claim 1 , wherein the depth map is generated based on the images received in the different pixel regions.

4 . The camera of claim 1 , wherein the sensor is sensitive to non-visible light.

5 . The camera of claim 1 , wherein at least one of the lenses is convex.

6 . A method of capturing images of an object using a camera comprising an image sensor having an array of light-sensing pixels and a plurality of lenses, each lens focusing light on one of pixel regions of the image sensor, the method comprising:
focusing light reflected from the object onto a plurality of different pixel regions of the image sensor; and capturing with the camera a plurality of images of the object, each image associated with one of the pixel regions of the image sensor; generating a three-dimensional (3D) image of the object based on the images; generating a depth map that calculates depth values for pixels of at least the object, based on the images received in the different pixel regions; detecting 3D motion of the object using the depth values; creating a 3D model of the object based on the 3D image; and tracking 3D motion of the object based on the 3D model.

7 . The method of claim 6 , further including:
acquiring additional images of the object as the object moves through space; analyzing the additional images to generate additional three-dimensional images of the object; and updating the 3D model to reflect 3D motion of the object based upon the additional three-dimensional (3D) images.

8 . The method of claim 6 , wherein the 3D model includes voxels.

9 . The method of claim 6 , wherein the camera captures images using non-visible light.

10 . The method of claim 6 , wherein reflected light is focused using at least one convex lens.

11 . An image capture and analysis system comprising:
a camera oriented toward a field of view, the camera comprising:
a single image sensor including an array of uniformly and continuously spaced light-sensing pixels;
a plurality of lenses that focus light reflected from an object onto a plurality of different pixel regions of the image sensor, each lens focusing light on a different one of the pixel regions; and
a processor for analyzing the pixel regions to generate a three-dimensional (3D) image of the object through a plurality of images successively obtained with the image sensor, generating a depth map that calculates depth values for pixels of at least the object, based on the images received in the different pixel regions, detecting 3D motion of the object using the depth values, creating a 3D model of the object based on the 3D image and tracking 3D motion of the object based on the 3D model.

12 . The system of claim 11 , wherein the processor further performs:
acquiring additional images of the object as the object moves through space; analyzing the additional images to generate additional three-dimensional images of the object; and updating the 3D model to reflect 3D motion of the object based upon the additional three-dimensional (3D) images.

13 . The system of claim 11 , wherein the 3D model includes voxels.

14 . The system of claim 11 , wherein the camera captures images using non-visible light.

15 . The system of claim 11 , wherein reflected light is focused using at least one convex lens.

16 . A method of image capture and analysis for use with a camera oriented toward a field of view, the camera comprising a single image sensor including an array of uniformly and continuously spaced light-sensing pixels and a plurality of lenses that focus light reflected from an object onto a plurality of different pixel regions of the image sensor, each lens focusing light on a different one of the pixel regions, the method comprising:
illuminating the field of view with at least one light source disposed on a same side of the field of view as the camera; operating the camera to capture a plurality of images, each image associated with one of the pixel regions of the image sensor, at a time when the light source is simultaneously operative to illuminate the field of view; analyzing the pixel regions to generate a three-dimensional (3D) image of the object through a plurality of images successively obtained with the image sensor; generating a depth map that calculates depth values for pixels of at least the object, based on the images received in the different pixel regions; detecting 3D motion of the object using the depth values; creating a 3D model of the object based on the 3D image; and tracking 3D motion of the object based on the 3D model.

17 . The method of claim 16 , further including:
acquiring additional images of the object as the object moves through space; analyzing the additional images to generate additional three-dimensional images of the object; and updating the 3D model to reflect 3D motion of the object based upon the additional three-dimensional (3D) images.

18 . The method of claim 16 , wherein the 3D model includes voxels.

19 . The method of claim 16 , wherein the camera captures images using non-visible light.

20 . The method of claim 16 , wherein reflected light is focused using at least one convex lens.

Assignees

• Leap Motion Inc

Inventors

• Holz David

Classifications

• H04N13/275

  from three-dimensional [3D] object models, e.g. computer-generated stereoscopic image signals · CPC title

• H04N13/271

  wherein the generated image signals comprise depth maps or disparity maps · CPC title

• H04N2213/001

  Constructional or mechanical details · CPC title

• G06T7/285

  using a sequence of stereo image pairs · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title