Optical artificial neural network intelligent chip, preparation method thereof and intelligent processing apparatus

Patent metadata

Field	Value
Publication number	US-12346799-B2
Application number	US-202117377217-A
Country	US
Kind code	B2
Filing date	Jul 15, 2021
Priority date	Feb 8, 2021
Publication date	Jul 1, 2025
Grant date	Jul 1, 2025

Abstract

An optical artificial neural network intelligent chip, a preparation method thereof and an intelligent processing apparatus. The optical filter layer serves as the input layer of the artificial neural network. The image sensor serves as the linear layer of the artificial neural network. The filtering effect of the optical filter layer on the incident light entering the optical filter layer serves as the connection weight from the input layer to the linear layer, such that the related functions of the input layer and the linear layer in the artificial neural network are achieved by the optical filter layer and the image sensor in the intelligent chip in a hardware manner, and no complicated signal processing and algorithm processing corresponding to the input layer and the linear layer need to be performed in the subsequent artificial neural network intelligent processing using the intelligent chip.

First claim

The invention claimed is:

1. An optical artificial neural network intelligent chip, comprising:
an optical filter layer,
an image sensor, and
a processor;
wherein the optical filter layer corresponds to an input layer and a connection weight from the input layer to a linear layer of an artificial neural network; the image sensor corresponds to the linear layer of the artificial neural network; and the processor corresponds to a nonlinear layer and an output layer of the artificial neural network;
the optical filter layer is disposed on the surface of a photosensitive area of the image sensor, the optical filter layer comprises an optical modulation structure, and is configured to modulate, with different spectra, incident light which is incident to different location points of the optical modulation structure respectively by the optical modulation structure, to obtain information carried in the incident light corresponding to different location points on the surface of the photosensitive area; the information carried in the incident light comprises light intensity distribution information, spectral information, angle information of the incident light and phase information of the incident light;
the image sensor is configured to convert information carried in the incident light corresponding to different location points after being modulated by the optical filter layer into electric signals corresponding to the different location points, and send the electric signals corresponding to the different location points to the processor, wherein the electrical signals are image signals modulated by the optical filter layer; and
the processor is configured to perform fully connected processing and nonlinear activation processing on the electrical signals corresponding to the different location points to obtain output signals of the artificial neural network.

2. The optical artificial neural network intelligent chip of claim 1 , wherein the optical artificial neural network intelligent chip is used for intelligent processing tasks of a target object; the intelligent processing tasks comprise at least one or more of an intelligent perception task, an intelligent recognition task and an intelligent decision task;
reflected light, transmitted light and/or radiation light of the target object enters a trained optical artificial neural network intelligent chip to obtain intelligent processing results of the target object; the intelligent processing results comprise at least one or more of an intelligent perception result, an intelligent recognition result and/or an intelligent decision result;
wherein the trained optical artificial neural network intelligent chip refers to an optical artificial neural network intelligent chip including a trained optical modulation structure, an image sensor and a processor; the trained optical modulation structure, the image sensor and the processor refer to an optical modulation structure, an image sensor and an processor satisfying a training convergence condition obtained by training the optical artificial neural network intelligent chip containing different optical modulation structures, image sensors, and processors with different fully connected parameters and nonlinear activation parameters using input training samples and output training samples corresponding to the intelligent processing task.

3. The optical artificial neural network intelligent chip of claim 2 , wherein when the optical artificial neural network intelligent chip containing different optical modulation structures, image sensors, and processors with different fully connected parameters and nonlinear activation parameters is trained, the different optical modulation structures are designed and implemented in a computer optical simulation design manner.

4. The optical artificial neural network intelligent chip of claim 1 , wherein the optical modulation structure in the optical filter layer comprises a regular structure and/or an irregular structure; and/or, the optical modulation structure in the optical filter layer comprises a discrete structure and/or a continuous structure.

5. The optical artificial neural network intelligent chip of claim 1 , wherein the optical filter layer has a single-layer structure or a multi-layer structure.

6. The optical artificial neural network intelligent chip of any one of claims 1 to 3 , wherein the optical modulation structure in the optical filter layer comprises a unit array composed of a plurality of micro-nano units, and each micro-nano unit corresponds to one or more pixels on the image sensor; each micro-nano unit has the same or different structure.

7. The optical artificial neural network intelligent chip of claim 6 , wherein the micro-nano unit comprises a regular structure and/or an irregular structure; and/or, the micro-nano unit comprises a discrete structure and/or a continuous structure.

8. The optical artificial neural network intelligent chip of claim 6 , wherein the micro-nano unit comprises a plurality of groups of micro-nano structure arrays, and each group of the micro-nano structure arrays has the same or different structure.

9. The optical artificial neural network intelligent chip of claim 8 , wherein each group of micro-nano structure array has broadband filtering or narrowband filtering functions.

10. The optical artificial neural network intelligent chip of claim 8 , wherein each group of micro-nano structure arrays is a periodic structure array or an aperiodic structure array.

11. The optical artificial neural network intelligent chip of claim 8 , wherein the plurality of groups of micro-nano structure arrays contained in the micro-nano unit has one or more groups of empty structures.

12. The optical artificial neural network intelligent chip of claim 8 , wherein the micro-nano unit has polarization-independent characteristics.

13. The optical artificial neural network intelligent chip of claim 12 , wherein the micro-nano unit has four-fold rotation symmetry.

14. The optical artificial neural network intelligent chip of claim 1 , wherein the optical filter layer is composed of one or more filter layers;
the filter layer is prepared from one or more of semiconductor materials, metal materials, liquid crystals, quantum dot materials, and perovskite materials; and the filter layer is a filter layer prepared from one or more of photonic crystals, metasurfaces, random structures, nanostructures, metal surface plasmon SPP micro-nano structure, and a tunable Fabry-Perot resonant cavity.

15. The optical artificial neural network intelligent chip of claim 14 , wherein the semiconductor materials comprise one or more of silicon, silicon oxide, silicon nitride, titanium oxide, a composite material mixed in a preset ratio and direct-gap semiconductor materials; and the nanostructure comprises one or more of nanodot two-dimensional materials, nanopillar two-dimensional materials, and nanowire two-dimensional materials.

16. The optical artificial neural network intelligent chip of claim 1 , wherein the optical filter layer has a thickness of 0.1λ˜10λ, wherein λ represents the center wavelength of the incident light.

17. The optical artificial neural network intelligent chip of claim 1 , wherein the image sensor is one or more of the following:
CMOS image sensor (CIS), charge coupled device (CCD), single photon avalanche diode (SPAD) array and focal plane photodetector array.

18. The optical artificial neural network intelligent chip of claim 1 , wherein the artificial neural network comprises a feedforward neural netwo

Assignees

• Univ Tsinghua

Inventors

• Cui Kaiyu
• Yang Jiawei
• Xiong Jian
• Huang Yidong
• Liu Fang
• Feng Xue
• Zhang Wei

Classifications

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/048

  Activation functions · CPC title

• G06N3/063

  using electronic means · CPC title

• G06N3/08

  Learning methods · CPC title