Image recognition system using a programmable photonic neural network

Patent metadata

Field	Value
Publication number	US-10217023-B1
Application number	US-201715622463-A
Country	US
Kind code	B1
Filing date	Jun 14, 2017
Priority date	Jun 14, 2017
Publication date	Feb 26, 2019
Grant date	Feb 26, 2019

Abstract

A system uses arrays of spatial light modulators (SLMs) connected to a processor and an image capture device. An image is input into a first array of SLMs. The processor determines if the output of the first array matches an image stored within a database. If a match is found, the processor outputs a stored image to an image processing system. If a match is not found the processor directs the output from the first array into an input of an array of SLMs adjacent to the first array. The determination step is iteratively performed for the remaining arrays of SLMs until a match is found or no arrays remain. If no arrays remain, the processor selects a stored image from the database and obtains user feedback from a user input system. The feedback is then stored in the database and associated with the n−1 array of SLMs.

First claim

I claim:

1. A method comprising:
providing n arrays of m spatial light modulators (SLMs) connected to both a processor and an image capture device, wherein the processor is connected to a database, wherein the arrays of SLMs are cascaded and have sequentially decreasing block ranks;
inputting an input image from the image capture device into a first array of SLMs of the arrays of SLMs, wherein the input image is represented by a matrix of bits;
determining, using the processor, if the output of the first array of SLMs matches a stored image stored within the database, wherein the output of the first array of SLMs is represented by a set of non-negative integer vectors created by the first array of SLMs by optically summing the matrix of bits using SLMs within the first array of SLMs, wherein a match is found by comparing the length of individual integer vectors within the set of non-negative integer vectors with vectors of stored images stored within the database, wherein if a match is found the processor outputs a stored image associated with the respective matching vectors of the stored image in the database to an image processing system, wherein if a match is not found the processor directs the set of non-negative integer vectors output from the first array of SLMs into an input of an array of SLMs adjacent to the first array of SLMs; and
iteratively performing the determining step for the remaining arrays of SLMs until either a match is found or no further arrays of SLMs remain, wherein if no further arrays of SLMs remain the processor randomly selects a stored image from the database and the processor obtains user feedback on the randomly-selected stored image from a user system connected to the processor, wherein the user feedback correctly identifies the recognition of the input image compared to the randomly-selected stored image, wherein the user feedback is stored by the processor in the database as being associated with the set of non-negative integer vectors input into the n−1 array of SLMs.

2. The method of claim 1 further comprising, prior to the step of inputting an input image from an image capture device connected to the array of SLMs into a first array of SLMs of the array of SLMs, pre-processing the input image using one or more of a feature detection algorithm, an edge detection algorithm, and an algorithm for processing hyperspectral imagery.

3. The method of claim 1 , wherein the database is segmented by vector length.

4. The method of claim 3 , wherein the step of determining, using the processor, if the output of the first array of SLMs matches a stored image stored within the database involves a search for a match with the vectors of stored images stored in the database by proceeding with the longest and most specific output vectors of the set of non-negative integer vectors created by the first array of SLMs to the shortest and most general output vectors of the set of non-negative integer vectors created by the first array of SLMs.

5. The method of claim 4 , wherein if more than one matches of output vectors of the set of non-negative integer vectors are made with the vectors of stored images stored in the database, the processor selects the match involving the longest and most specific output vectors of the set of non-negative integer vectors.

6. The method of claim 4 , wherein the search for a match with the vectors of stored images stored in the database is performed by one of a parallel matching algorithm, a bisection algorithm, and a linear search.

7. The method of claim 1 further comprising the step of, prior to the processor outputting a stored image associated with the respective matching vectors of the stored image in the database to an image processing system if a match is found, the processor receives user feedback on the matched stored image, wherein if the user feedback on the matched stored image indicates an error, then the processor replaces the semantics associated with the vectors of the matched stored image with semantics associated with the user feedback.

8. The method of claim 1 further comprising the step of, prior to the processor outputting a stored image associated with the respective matching vectors of the stored image in the database to an image processing system if a match is found, the processor receives input from a case-based reasoning system for associative memory.

9. The method of claim 1 further comprising the step of, prior to the processor outputting a stored image associated with the respective matching vectors of the stored image in the database to an image processing system if a match is found, the processor receives input from a case-based reasoning system for scalable generalization.

10. A system comprising:
an image capture device;
a plurality of n arrays of m spatial light modulators (SLMs) connected to the image capture device, wherein the arrays of SLMs are cascaded and have sequentially decreasing block ranks;
a processor connected to the arrays of SLMs;
a database connected to the processor, the database having a plurality of stored images stored therein;
an image processing device connected to the processor; and
a user system connected to the processor,
wherein the image capture device is configured to input an input image into a first array of SLMs of the arrays of SLMs, wherein the input image is represented by a matrix of bits, wherein the processor is configured, via a set of implementable instructions stored therein, to determine if the output of the first array of SLMs matches a stored image stored within the database, wherein the output of the first array of SLMs is represented by a set of non-negative integer vectors created by the first array of SLMs by optically summing the matrix of bits using SLMs within the first array of SLMs, wherein the processor is configured to determine a match by comparing the length of individual integer vectors within the set of non-negative integer vectors with vectors of stored images stored within the database, wherein if a match is found the processor is configured to output a stored image associated with the respective matching vectors of the stored image in the database to the image processing device.

11. The system of claim 10 , wherein if a match is not found, the processor is further configured to direct the set of non-negative integer vectors output from the first array of SLMs into an input of an array of SLMs adjacent to the first array of SLMs.

12. The system of claim 11 , wherein the processor is further configured to iteratively perform the determining step for the remaining arrays of SLMs until either a match is found or no further arrays of SLMs remain, wherein if no further arrays of SLMs remain the processor is configured to randomly select a stored image from the database and the processor is configured to obtain user feedback on the randomly-selected stored image from the user system, wherein the user feedback correctly identifies the recognition of the input image compared to the randomly-selected stored image.

13. The system of claim 12 , wherein the processor is further configured to store the user feedback in the database as being associated with the set of non-negative integer vectors input into the n−1 array of SLMs.

14. The system of claim 10 , wherein each SLM contained within the arrays of SLMs comprise frequency modulated SLMs.

15. The system of claim 10 , wherein each SLM contained within the arrays of SLMs comprise amplitude modulated SLMs.

16. The system of claim 10 , wherein each SLM contained within the arrays of SLMs comprise SLMs with four input por

Assignees

• Us Navy
• Us Navy

Inventors

• Rubin Stuart H

Classifications

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2413

  based on distances to training or reference patterns · CPC title

• G06V10/24

  Aligning, centring, orientation detection or correction of the image · CPC title

• G06V10/32

  Normalisation of the pattern dimensions · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title