Hybrid machine learning systems

Patent metadata

Field	Value
Publication number	US-11055824-B2
Application number	US-201815866889-A
Country	US
Kind code	B2
Filing date	Jan 10, 2018
Priority date	Jul 14, 2015
Publication date	Jul 6, 2021
Grant date	Jul 6, 2021

Abstract

A machine learning system for processing image data obtained from an image sensor is provided. The system includes a front end comprising one or more hard-coded filters, each of the one or more hard-coded filters being arranged to perform a set task. The system includes a neural network arranged to receive and process output from the front end. The one or more hard-coded filters include one or more hard-coded noise compensation filters that are hard-coded to compensate for a noise profile of the image sensor from which the image data is obtained. A method of processing image data in a machine learning system is also provided. A system for processing image data is provided.

First claim

What is claimed is:

1. A machine learning system for processing image data obtained from an image sensor, the machine learning system comprising:
a front end comprising one or more hard-coded segmentation operators to partition an image into a plurality of non-overlapping regions, and one or more hard-coded filters, each of the one or more hard-coded filters being arranged to perform a set task; and
a neural network arranged to receive and process output from the front end,
wherein the one or more hard-coded filters include one or more hard-coded non-adaptive noise compensation filters that are hard-coded to compensate for a noise profile of the image sensor from which the image data is obtained, and wherein the machine learning system is arranged to normalize a response of the one or more hard-coded non-adaptive noise compensation filters based on the noise profile of the image sensor from which the image data is obtained.

2. The machine learning system according to claim 1 , wherein the front end comprises one or more color feature extractors.

3. The machine learning system according to claim 2 , the machine learning system being configured to provide input pixels obtained from the image sensor to the one or more color feature extractors via one or more input pixel streams.

4. The machine learning system according to claim 3 , the one or more color feature extractors being configured to convert the input pixels in the one or more input pixel streams into YUV color space components without clipping.

5. The machine learning system according to claim 3 , the one or more color feature extractors being configured to extract hue and saturation components from the one or more input pixel streams.

6. The machine learning system according to claim 1 , wherein the front end comprises one or more Gabor filters.

7. The machine learning system according to claim 1 , wherein the front end comprises one or more correlation filters.

8. The machine learning system according to claim 7 , wherein the one or more correlation filters include one or more sharpening filters, one or more brightening filters, one or more edge detection filters, one or more texture feature extraction filters, one or more auto-white balance filters, one or more color extraction filters, and/or one or more further noise compensation filters.

9. The machine learning system according to claim 1 , comprising one or more multiscale decomposition engines arranged to receive the image data from the image sensor and to output data to the front end.

10. The machine learning system according to claim 1 , comprising one or more multiscale decomposition engines arranged to receive data from the front end.

11. The machine learning system according to claim 10 , wherein the one or more multiscale decomposition engines arranged to receive data from the front end are arranged to output data to the neural network.

12. The machine learning system according to claim 1 , wherein the image data is raw image data.

13. The machine learning system according to claim 1 , wherein the machine learning system is arranged to process the image data in real time.

14. The machine learning system according to claim 1 , wherein the neural network is a convolutional neural network.

15. The machine learning system according to claim 1 , wherein the machine learning system comprises at least one of: an image recognition system or a facial recognition system.

16. The machine learning system according to claim 1 , wherein the machine learning system is configured to operate as a classification and captioning system.

17. The machine learning system according to claim 1 , wherein each of the plurality of non-overlapping regions are homogeneous in one or more image features.

18. A method of processing image data in a machine learning system, the method comprising:
processing, in a front end of the machine learning system, image data obtained from an image sensor, the front end comprising one or more hard-coded segmentation operators to partition an image into a plurality of non-overlapping regions, and one or more hard-coded filters, each of the one or more hard-coded filters being arranged to perform a set task, wherein the one or more hard-coded filters include a hard-coded non-adaptive noise compensation filter that is hard-coded to compensate for a noise profile of the image sensor from which the image data is obtained;
normalizing a response of the hard-coded non-adaptive noise compensation filter based on the noise profile of the image sensor from which the image data is obtained; and
receiving and processing output from the front end in a neural network part of the machine learning system.

19. A system for processing image data, the system comprising:
a front end comprising one or more hard-coded segmentation operators to partition an image into a plurality of non-overlapping regions, and one or more hard-coded non-adaptive filters hard-coded to compensate for a noise profile of one or more image sensors from which the image data is obtained, wherein the system is arranged to normalize a response of the one or more hard-coded non-adaptive filters based on the noise profile; and
a back end comprising one or more neural networks adapted to receive and process output from the front end.

Assignees

• Apical Ltd

Inventors

• Romanenko Ilya

Classifications

• G06V10/80

  Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level (multimodal speaker identification or verification G10L17/10) · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2451

  linear, e.g. hyperplane · CPC title

• G06F18/25

  Fusion techniques · CPC title