Integrated three-dimensional vision sensor

Patent metadata

Field	Value
Publication number	US-9532030-B2
Application number	US-201214367911-A
Country	US
Kind code	B2
Filing date	Dec 13, 2012
Priority date	Dec 22, 2011
Publication date	Dec 27, 2016
Grant date	Dec 27, 2016

Abstract

A three-dimensional scene sensor comprises: a deformable optical system modifying focal distance by control signal, optics imaging the scene by analog image sensor for depths corresponding to distances; the image sensor comprising a matrix of pixels grouped into sub-matrices of macro-pixels being a sub-assembly of pixels, each macro-pixel operating independently for acquisition and reading of data; a matrix of elementary processors, each macro-pixel directly connected to a dedicated processor wherein pixel data for the macro-pixel are transmitted and processed by the processor, each processor carries out, for each pixel, local processing operations calculating depth information for macro-pixel, the processors operating in parallel and independently such that the depth information is processed and calculated in parallel over all macro-pixels of the image sensor, the processors connected to at least one processing unit allowing calculations using high-level input data, calculated starting from the pixel data directly produced by the image sensor.

First claim

The invention claimed is:

1. A three-dimensional scene sensor comprising:
a deformable optical system allowing a modification of a focal distance as a function of a control signal,
said optical system imaging said scene on an analog image sensor for a plurality of depths corresponding to a plurality of values of focal distance,
said analog image sensor carrying out one image acquisition for each value of focal distance,
the images being analyzed in order to determine depth information,
said analog image sensor comprising a matrix of pixels, said pixels being grouped into sub-matrices called macro-pixels composed of a sub-assembly of pixels, each macro-pixel operating independently of its neighbors for the acquisition and a reading of data from said pixels for each portion of image, one portion of image being respectively associated with one macro-pixel,
a matrix of elementary processors, each macro-pixel being directly connected via an interface to a dedicated elementary processor in such a manner that the pixel data relating to said pixels belonging to said macro-pixel are transmitted and processed by said processor,
each of said elementary processors comprising a local memory, a neighborhood manager and a plurality of local processing units in such a manner as to carry out, for each of the pixels of the macro-pixel to which said processor is connected and for each portion of image, local processing operations taking into account the neighboring pixels, said local processing operations allowing depth information to be calculated for said macro-pixel,
said processors operating in parallel and independently of one another such that for each portion of image the depth information is processed and calculated in parallel and independently over all of the macro-pixels of said image sensor,
each elementary processor being capable of modifying certain parameters of the image sensor corresponding to the associated macro pixel,
said processors being designed to be connected to at least one processing unit allowing calculations to be carried out using high-level input data, obtained starting from depth information calculated by said elementary processors.

2. The sensor as claimed in claim 1 wherein said deformable optical system, said image sensor and said processor matrix form a stack.

3. The sensor as claimed in claim 1 wherein said deformable optical system allows a modification of its optical axis as a function of the control signal.

4. The sensor as claimed in claim 1 wherein the elementary processors are designed to communicate with their near neighbors so as to exchange depth information.

5. The sensor as claimed in claim 1 each wherein macro-pixel comprises a plurality of columns of pixels, each column being connected to a circuit of analog/digital converter type for reading the pixel data of each said column.

6. The sensor as claimed in claim 1 wherein said local memory of a macro-pixel contains the depth information updated at each new acquisition by said sensor of an image portion.

7. The sensor as claimed in claim 6 wherein depth information stored in said local memory comprises a value of best sharpness for a corresponding macro-pixel and a value of an index of a focal length for which the best value of sharpness of the corresponding macro-pixel is obtained.

8. The sensor as claimed in claim 7 wherein said depth information stored in said local memory comprises a depth value.

9. The sensor as claimed in claim 1 wherein the processing unit furthermore comprises a shared memory connected to both the matrix of elementary processors and to an assembly of multi-core processors.

10. The sensor as claimed in claim 1 wherein said high-level input data comprise a depth map.

11. The sensor as claimed in claim 1 wherein said high-level input data comprise a sharp image at all points.

12. The sensor as claimed in claim 1 wherein the processing unit is capable of a feedback action on said deformable optical system as a function of a result of said calculations performed by said processing unit.

13. The sensor as claimed in claim 1 further comprising a component designed to communicate the depth information to a remote system.

14. The sensor as claimed in claim 1 wherein the depth information is obtained by means of an iterative calculation method based on a determination of the focal distance of said deformable optical system allowing the best sharpness to be obtained, said calculation method being carried out locally within said processor matrix and in parallel for each macro-pixel within said processor matrix.

15. The sensor as claimed in claim 14 wherein said calculation method is carried out locally within said processor matrix.

16. The sensor as claimed in claim 14 wherein said calculation method allows an update at each iteration of the depth information.

17. A system for capturing a three-dimensional scene comprising at least one sensor as claimed in claim 1 and furthermore comprising a display device capable of displaying a three-dimensional image of the captured scene.

Assignees

• Commissariat Energie Atomique

Inventors

• Kolar Anthony
• Duranton Marc

Classifications

• H04N25/443

  by reading pixels from selected two-dimensional [2D] regions of the array, e.g. for windowing or digital zooming · CPC title

• H04N5/3454

  Electricity · mapped topic

• H04N5/2254

  Electricity · mapped topic

• G06T1/20

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• H04N13/0235 Primary

  Electricity · mapped topic