Sensor noise profile

Patent metadata

Field	Value
Publication number	US-9892517-B2
Application number	US-201514973937-A
Country	US
Kind code	B2
Filing date	Dec 18, 2015
Priority date	Dec 19, 2014
Publication date	Feb 13, 2018
Grant date	Feb 13, 2018

Abstract

The invention relates to feature extraction technique based on edge extraction. It can be used in computer vision systems, including image/facial/object recognition systems, scene interpretation, classification and captioning systems. A model or profile of the noise in the sensor is used to improve feature extraction or object detection on an image from a sensor.

First claim

The invention claimed is:

1. A method of extracting a feature from an image, the method comprising performing, in a processor or circuitry, steps of:
receiving a digital image from a sensor;
demosaicing the digital image;
prior to demosaicing the digital image, taking an input image from the digital image for feature extraction; and
performing feature extraction on the input image, wherein the step of performing feature extraction comprises:
using a model or profile of noise comprising a variance of the noise in the sensor in defined areas of the input image to normalize a response of a feature extraction algorithm;
applying the feature extraction algorithm to the input image; and
for each pixel of the input image;
calculating an edge response for an orientation, and
normalizing the edge response for the orientation by taking into account the variance in the noise in the sensor.

2. The method of claim 1 , wherein the feature is an edge.

3. The method of claim 2 , wherein the feature extraction is based on detection of the edge.

4. The method of claim 1 , wherein the feature is a local binary pattern.

5. The method of claim 1 , wherein the method is implemented in a system that is not part of, does not use, or is not downstream of, an image processing pipeline.

6. The method of claim 1 , wherein the method operates operating in a RAW domain with linear data.

7. The method of claim 1 , wherein the edge response is calculated by convoluting a filter kernel with an intensity of the input image.

8. The method of claim 7 , wherein the filter kernel is a Gabor filter kernel or a Convolutional Neural Network filter kernel.

9. The method of claim 1 , in which the normalizing of the edge response for an orientation ∝ is calculated for each pixel (x,y) in the input image from:

E
norm
∝

⁡

(

x
,
y

)

=

E
∝

⁡

(

x
,
y

)

Σ

i
,

k
⁢

∈
K

⁢

σ
⁡

(

x
+
i

,

y
+
k

)

×

G
K

⁡

(

x
+
i

,

y
+
k

)

wherein the response E^∝(x,y) is calculated from
E ∝ ( x,y )=|Σ i,kεK G sin ∝ (x+i,y+k) ×I ( x+i,y+k )|+|Σ i,kεK G cos ∝ (x+i,y+k) ×I ( x+i,y+k )|

10. The method of claim 1 , wherein the input image is RAW image sensor data.

11. The method of claim 1 , wherein the edge response is fed into a linear classifier comprising a Support Vector Machine or into a classification layer of a Convolutional Neural Network.

12. The method of claim 1 , wherein the method is implemented in real-time.

13. The method of claim 1 , wherein the method operates as a part of a computer vision system, applied to posture detection, people detection, object detection in the digital image.

14. The method of claim 1 , wherein the method is utilized in one of the following: Smartphone; Computer vision systems; Objects recognition systems; Human detection systems; Facial recognition systems; Scene interpretation systems; Image classification systems; Image captioning systems; Autonomous vehicle computer vision systems; Robotics computer vision systems.

15. The method of claim 1 , wherein the method is implemented in an embedded hardware block.

16. Image processing hardware configured to:
receive a digital image from a sensor;
demosaic the digital image;
prior to demosaicing the digital image, take an input image from the digital image for feature extraction; and
perform feature extraction on the input image, wherein the step of performing feature extraction comprises:
using a model or profile of noise comprising a variance of the noise in the sensor in defined areas of the input image to normalize a response of a feature extraction algorithm;
applying the feature extraction algorithm to the input image; and
for each pixel of the input image;
calculating an edge response for an orientation, and
normalizing the edge response for the orientation by taking into account the variance in the noise in the sensor.

17. A device including image processing hardware configured to:
receive a digital image from a sensor;
demosaic the digital image;
prior to demosaicing the digital image, take an input image from the digital image for feature extraction; and
perform feature extraction on the input image, wherein the step of performing feature extraction comprises:
using a model or profile of noise in the sensor to normalize a response of a feature extraction algorithm;
applying the feature extraction algorithm to the input image; and
for each pixel of the input image;
calculating an edge response for an orientation, and
normalizing the edge response for the orientation by taking into account the variance in the noise in the sensor.

18. The device of claim 17 , wherein the device comprises or is a part of one of the following: Smartphone; Computer vision system; Objects recognition system; Human detection system; Facial recognition system; Scene interpretation system; Image classification system; Image captioning system; Autonomous vehicle computer vision system; Robotics computer vision system.

Assignees

• Apical Ltd
• Apical Ltd

Inventors

• Romanenko Ilya

Classifications

• G06V10/467

  Encoded features or binary features, e.g. local binary patterns [LBP] · CPC title

• G06V10/449

  Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06F2218/04

  Denoising · CPC title