Sensor noise profile

Patent metadata

Field	Value
Publication number	US-2016180541-A1
Application number	US-201514973937-A
Country	US
Kind code	A1
Filing date	Dec 18, 2015
Priority date	Dec 19, 2014
Publication date	Jun 23, 2016
Grant date	—

Abstract

The invention relates to feature extraction technique based on edge extraction. It can be used in computer vision systems, including image/facial/object recognition systems, scene interpretation, classification and captioning systems. A model or profile of the noise in the sensor is used to improve feature extraction or object detection on an image from a sensor.

First claim

1 . A method of extracting a feature from an image, comprising the processor or circuitry implemented steps of:
(a) providing a digital image from a sensor; (b) using a model or profile of the noise in the sensor to improve feature extraction or object detection on the image.

2 . Method of claim 1 where the feature is an edge.

3 . Method of claim 1 where the feature is a local binary pattern.

4 . Method of claim 1 where the model or profile of the sensor noise is used to normalize feature extraction response.

5 . Method of claim 1 where the variance of the sensor noise in defined areas of the image are used to normalize feature extraction response.

6 . Method of claim 1 where the feature extraction or object detection is based on edge detection.

7 . Method of claim 1 and that includes the steps of, for each pixel of the input image, calculating an edge response for an orientation, and normalising the edge response for an orientation by taken into account the noise variance.

8 . Method of claim 1 , implemented in a system that is not part of, does not use, or is not downstream of, an image processing pipeline.

9 . Method of claim 1 , operating in the RAW domain with linear data.

10 . Method of claim 1 , in which the edge response is calculated by convoluting a filter kernel with the intensity of the image.

11 . Method of claim 1 , in which the filter is a Gabor filter or a CNN filter.

12 . Method of claim 1 , in which the normalization of the edge response for an orientation a is calculated for each pixel (x,y) in the image from:

E
norm
∝



(

x
,
y

)

=

E
∝



(

x
,
y

)

Σ

i
,

k


∈
K



σ


(

x
+
i

,

y
+
k

)

×

G
K



(

x
+
i

,

y
+
k

)

wherein the response E ∝ (x, y) is calculated from
E ∝ ( x, y )=|Σ i,k ∈K G sin ∝ (x+i,y+k) ×I ( x+i, y+k )|+|Σ i,k ∈K G cos ∝ (x+i,y+k) ×I ( x+i, y+k )|

13 . Method of claim 1 , in which the input image is the RAW image sensor data.

14 . Method of claim 1 , in which the image edge response is fed into a linear classifier such as an SVM or into a classification layer of a CNN.

15 . Method of claim 1 , implemented in real-time.

16 . Method of claim 1 , operating as a computer vision system, applied to posture detection, people detection, object detection.

17 . Method of claim 1 when used in one of the following: Smartphone; Computer vision systems; Objects recognition systems; Human detection systems; Facial recognition systems; Scene interpretation systems; Image classification systems; Image captioning systems; Autonomous vehicle computer vision systems; Robotics computer vision systems.

18 . Method of claim 1 when implemented in embedded hardware, such as a hardware block.

19 . Image processing hardware configured to receive a digital image from a sensor and use a model or profile of the noise in the sensor to improve feature extraction or object detection on the image.

20 . A device including image processing hardware configured to receive a digital image from a sensor and use a model or profile of the noise in the sensor to improve feature extraction or object detection on the image.

21 . The device of claim 20 , being or including one of the following:
Smartphone; Computer vision system; Objects recognition system; Human detection system; Facial recognition system; Scene interpretation system; Image classification system; Image captioning system; Autonomous vehicle computer vision system; Robotics computer vision system.

Assignees

• Apical Ltd

Inventors

• Romanenko Ilya

Classifications

• G06V10/467

  Encoded features or binary features, e.g. local binary patterns [LBP] · CPC title

• G06V10/449

  Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06F2218/04

  Denoising · CPC title