Apparatus and method for generating local binary patterns (LBPS)

Patent metadata

Field	Value
Publication number	US-10129495-B2
Application number	US-201615081642-A
Country	US
Kind code	B2
Filing date	Mar 25, 2016
Priority date	Mar 25, 2016
Publication date	Nov 13, 2018
Grant date	Nov 13, 2018

Abstract

Techniques for direct local binary pattern (LBP) generation are presented. An image sensor for LBP generation includes a variable reference signal generator and a sensor pixel array that can generate events based on optical signals on the sensor pixel array and a reference level from the variable reference signal generator. The image sensor also includes an address encoder that can encode the addresses of the sensor pixels that generate events, and a binary image generator that can create a binary image based on the addresses of the sensor pixels that generate the events at the reference level. The image sensor may also include a local binary pattern generator configured to determine local binary pattern labels for image pixels whose binary value changes from a first binary image at a first reference level to a subsequent second binary image at a next reference level.

First claim

What is claimed is:

1. A method comprising:
receiving a variable reference signal at a plurality of sensor pixels, wherein the variable reference signal changes monotonically within a plurality of reference levels, and wherein a same reference level is received at the plurality of sensor pixels at a given time; and
at each reference level of the plurality of reference levels:
converting, by each sensor pixel of the plurality of sensor pixels, an optical signal on each sensor pixel of the plurality of sensor pixels into an electrical signal;
generating an event at each sensor pixel on which a condition based on a comparison between the electrical signal and the reference level is met;
encoding an address of each sensor pixel that generates the event; and
creating a binary image for the reference level based on the address of each sensor pixel that generates the event.

2. The method of claim 1 , wherein the condition based on the comparison between the electrical signal and the reference level is met when the electrical signal is one of greater than, no less than, less than, or no greater than the reference level.

3. The method of claim 1 , further comprising:
when a binary value of an image pixel changes from a first binary image for a first reference level to a subsequent second binary image for a next reference level, determining a local binary pattern label for the image pixel based on binary values of neighboring image pixels of the image pixel in the first binary image or the subsequent second binary image.

4. The method of claim 3 , further comprising:
adding the local binary pattern label for the image pixel to a local binary pattern histogram.

5. The method of claim 1 , wherein the variable reference signal is an output of a single-slope ramp signal generator.

6. The method of claim 5 , wherein the variable reference signal has a rising slope or a falling slope.

7. The method of claim 1 ,
wherein the plurality of sensor pixels form a two-dimensional array; and
wherein the address of each sensor pixel includes a column address and a row address.

8. The method of claim 1 , wherein creating the binary image based on the address of each sensor pixel that generates the event includes:
assigning a “1” to each image pixel corresponding to each sensor pixel that generates the event, and assigning a “0” to each image pixel corresponding to each sensor pixel that does not generate the event; or
assigning a “1” to each image pixel corresponding to each sensor pixel that generates the event at a rate equal to or greater than a threshold rate, and assigning a “0” to each image pixel corresponding to each sensor pixel that generates the event at a rate less than the threshold rate.

9. The method of claim 1 , wherein the event is a pulse.

10. An image sensor comprising:
a sensor pixel array including a plurality of sensor pixels, wherein each sensor pixel in the sensor pixel array includes a sensing element and a comparator, a signal input of the comparator coupled to the sensing element;
a global reference signal input port coupled to a reference input of the comparator of each sensor pixel, wherein the global reference signal input port is configured to receive a global reference signal varying monotonically among a plurality of reference levels, and wherein a same reference level is received at the reference inputs of the comparators of the plurality of sensor pixels at a given time;
an address encoder coupled to an output of the comparator of each sensor pixel in the sensor pixel array, the address encoder configured to encode an address of each sensor pixel that generates an event at the output of the comparator; and
a binary image generator configured to create a binary image at each reference level of the plurality of reference levels based on the address of each sensor pixel that generates the event at the each reference level.

11. The image sensor of claim 10 , further comprising:
a local binary pattern generator configured to determine local binary pattern labels for image pixels whose binary value changes from a first binary image at a first reference level to a subsequent second binary image at a next reference level.

12. The image sensor of claim 11 , wherein the local binary pattern generator is further configured to add the local binary pattern labels to a local binary pattern histogram.

13. The image sensor of claim 10 , further comprising:
a single-slope ramp signal generator coupled to the global reference signal input port.

14. The image sensor of claim 13 , wherein the single-slope ramp signal generator is configured to generate a rising slope or a falling slope.

15. The image sensor of claim 10 ,
wherein the sensor pixel array is a two-dimensional pixel array; and
wherein the address encoder includes:
a column address encoder configured to encode a column address of each sensor pixel that generates the event at the output of the comparator; and
a row address encoder configured to encode a row address of each sensor pixel that generates the event at the output of the comparator.

16. The image sensor of claim 10 , wherein the binary image generator is configured to create the binary image based on the address of each sensor pixel that generates the event at each reference level by:
assigning a “1” to each image pixel corresponding to each sensor pixel that generates the event, and assigning a “0” to each image pixel corresponding to each sensor pixel that does not generate the event; or
assigning a “1” to each image pixel corresponding to each sensor pixel that generates the event at a rate equal to or greater than a threshold rate, and assigning a “0” to each image pixel corresponding to each sensor pixel that generates the event at a rate less than the threshold rate.

17. The image sensor of claim 10 , wherein the event is a pulse.

18. The image sensor of claim 10 , wherein the plurality of sensor pixels are complementary metal-oxide-semiconductor (CMOS) image sensors.

19. The image sensor of claim 10 , wherein each sensor pixel in the sensor pixel array further includes a correlated double sampling (CDS) circuit configured to reduce noise.

20. An apparatus for local binary pattern generation, comprising:
means for receiving a variable reference signal at a plurality of sensor pixels, wherein the variable reference signal changes monotonically within a plurality of reference levels, and wherein a same reference level is received at the plurality of sensor pixels at a given time;
means for converting an optical signal on each sensor pixel of the plurality of sensor pixels into an electrical signal;
means for generating an event at each sensor pixel on which a condition based on a comparison between the electrical signal and a reference level of the plurality of reference levels is met;
means for encoding an address of each sensor pixel that generates the event; and
means for creating a binary image for each reference level of the plurality of reference levels based on the address of each sensor pixel that generates the event at the each reference level.

21. The apparatus of claim 20 , wherein the condition is met when the electrical signal is one of greater than, no less than, less than, or no greater than the reference level.

22. The apparatus of claim 20 , further comprising:
means for, when a binary value of an image pixel changes from a first binary image for a first reference level to a subsequent second binary image

Assignees

• Qualcomm Inc

Inventors

• Fu Zhengming
• Lu Shan
• Ling Jing

Classifications

• H04N25/76

  Addressed sensors, e.g. MOS or CMOS sensors · CPC title

• G06V10/955 Primary

  using specific electronic processors · CPC title

• G06V10/467

  Encoded features or binary features, e.g. local binary patterns [LBP] · CPC title

• G06K9/4604

  Physics · mapped topic

• G06F1/3234

  Power saving characterised by the action undertaken · CPC title