Methods for enabling in-field selection of near-sensor digital imaging functions

Patent metadata

Field	Value
Publication number	US-10771722-B2
Application number	US-201816145272-A
Country	US
Kind code	B2
Filing date	Sep 28, 2018
Priority date	Sep 29, 2017
Publication date	Sep 8, 2020
Grant date	Sep 8, 2020

Abstract

An imaging device is often paired with a readout integrated circuit (ROIC), which provides processing and data transfer functionality. The circuitry of a ROIC is typically specialized to meet the requirements of an application, which limits the ROIC to a few modes of operation and restricts compatibility to only certain types of imaging devices and applications. Furthermore, the circuitry supporting the processing functionality is limited due to size constraints on the ROIC. These shortcomings can be overcome with a field programmable imaging array (FPIA), which can be implemented as an integrated circuit combining customized ROIC sensor interface circuitry with field programmable gate array (FPGA) circuitry to enable post-fabrication definition of ROIC operational modes. An FPIA chip may form part of a three-chip stack that also includes an analog sensor interface chip for analog-to-digital conversion and an imaging device.

First claim

The invention claimed is:

1. A field programmable imaging array (FPIA) chip, comprising:
a substrate;
a plurality of macropixel elements, disposed on the substrate, to process digital data, each one of the macropixel elements including non-reconfigurable circuitry;
a plurality of field programmable gate array (FPGA) elements, disposed on at least a portion of the substrate in electronic communication with the plurality of macropixel elements, to receive processed digital data from the plurality of macropixel elements;
a plurality of reconfigurable interconnects, disposed on the substrate, to reconfigurably connect the plurality of macropixel elements to the plurality of FPGA elements; and
a plurality of deserializer elements, disposed on the substrate, to convert serial digital data from the plurality of macropixel elements to parallel digital data for processing by the plurality of FPGA elements.

2. The FPIA chip of claim 1 , wherein a first reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one FPGA element to each macropixel element.

3. The FPIA chip of claim 1 , wherein a second reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one macropixel element in the plurality of macropixel elements to a first deserializer element in the plurality of deserializer elements.

4. The FPIA chip of claim 3 , wherein a third reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one FPGA element in the plurality of FPGA elements to the first deserializer element.

5. The FPIA chip of claim 1 , further comprising:
a plurality of input/output (I/O) elements, disposed on the substrate, to at least one of receive a control signal or supply a signal output.

6. The FPIA chip of claim 5 , wherein a fourth reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one FPGA element to a first I/O element in the plurality of I/O elements.

7. A method of processing data with a field programmable imaging array (FPIA) chip, the FPIA chip comprising a substrate, macropixel elements disposed on the substrate, and field programmable gate array (FPGA) elements disposed on the substrate in electronic communication with the plurality of FPGA elements, the method comprising:
processing digital data with the macropixel elements;
transferring processed digital data from the macropixel elements to the FPGA elements;
reconfiguring connections between the macropixel elements and the FPGA elements; and
converting serial digital data from the macropixel elements to parallel digital data for processing by the FPGA elements.

8. The method of claim 7 , wherein reconfiguring connections between the macropixel elements and the FPGA elements comprises:
connecting a first macropixel element to a first FPGA element at a first time step; and
connecting the first macropixel element to a second FPGA element at a second time step.

9. The method of claim 7 , further comprising:
reconfigurably coupling a first macropixel element to a deserializer element that converts the serial digital data to parallel digital data.

10. The method of claim 9 , further comprising
reconfigurably connecting the deserializer element to a first FPGA element.

11. The method of claim 7 , further comprising:
reconfigurably connecting a first FPGA element to an input/output (I/O) element; and
receiving a control signal and/or supplying a signal output with the I/O element.

12. An apparatus, comprising:
a sensor array chip to generate analog data;
a sensor interface chip, having a first side disposed on a first side of the sensor array chip, to convert the analog data to digital data; and
a digital processing chip, having a first side disposed on a second side of the sensor interface chip, to process the digital data provided by the sensor interface chip, the digital processing chip comprising a plurality of deserializer elements to convert serial digital data to parallel digital data,
wherein a second reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one macropixel element to a first deserializer element in the plurality of deserializer elements.

13. The apparatus of claim 12 , wherein the digital processing chip comprises a field programmable imaging array (FPIA) chip, the FPIA chip comprising:
a plurality of macropixel elements to receive and/or generate the digital data from the sensor interface chip inputs and outputs; and
a plurality of field programmable gate array (FPGA) elements, in electronic communication with the plurality of macropixel elements, to process the digital data.

14. The apparatus of claim 13 , wherein a first macropixel element in the plurality of macropixel elements is configured to receive digital data representing light detected by a first sensor in the sensor array chip.

15. The apparatus of claim 14 , wherein the first macropixel is configured to be dynamically reconfigured to receive digital data representing light detected by a second sensor in the sensor array chip.

16. The apparatus of claim 15 , wherein the first macropixel element includes non-reconfigurable circuitry.

17. The apparatus of claim 13 , further comprising:
a plurality of reconfigurable interconnects, disposed on the FPIA chip, to reconfigurably connect the plurality of macropixel elements to the plurality of FPGA elements.

18. The apparatus of claim 17 , wherein a first reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one FPGA element to a first macropixel element in the plurality of macropixel elements.

19. The apparatus of claim 17 , further comprising:
a plurality of input/output (I/O) elements, disposed on the FPIA chip, to receive a control signal and provide a signal output.

20. The apparatus of claim 19 , wherein a fourth reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one FPGA element to a first I/O element in the plurality of I/O elements.

21. The apparatus of claim 12 , wherein a third reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one FPGA element to the first deserializer element.

Assignees

• Grossmann Peter J
• STAMPLIS Matthew
• Thurmer Kate
• Tyrrell Brian
• Frechette Jonathan
• Massachusetts Inst Technology

Inventors

• Grossmann Peter J
• STAMPLIS Matthew
• Thurmer Kate
• Tyrrell Brian
• Frechette Jonathan

Classifications

• H04N25/76 Primary

  Addressed sensors, e.g. MOS or CMOS sensors · CPC title

• H04N25/79

  Arrangements of circuitry being divided between different or multiple substrates, chips or circuit boards, e.g. stacked image sensors · CPC title

• H10F39/809 Primary

  of hybrid image sensors · CPC title

• H10F39/184

  Infrared image sensors · CPC title

• H04N25/78

  Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters · CPC title