Event vision sensors with event data compression, including event vision sensors with in-pixel event data compression, and associated systems, devices, and methods

Patent metadata

Field	Value
Publication number	US-12356093-B2
Application number	US-202318329378-A
Country	US
Kind code	B2
Filing date	Jun 5, 2023
Priority date	Jun 5, 2023
Publication date	Jul 8, 2025
Grant date	Jul 8, 2025

Abstract

Event vision sensors with event data compression (and associated systems, devices, and methods) are disclosed herein. In one embodiment, an event vision sensor includes a plurality of event vision pixels. Each event vision pixel of the plurality is configured to generate event data based on events indicated in incident light received from an external scene, and includes a compression circuit configured to compress the event data prior to readout of the event data from the event vision pixel. Each compression circuit can include a time aggregation circuit that is configured to track a number of the events detected by the corresponding event vision pixel over a specified timing window. The compressed event data can be read out from the event vision sensors and used to generate a pseudo-frame. The event vision sensor can optionally perform frame-level compression on the pseudo-frame.

First claim

What is claimed is:

1. An event vision pixel, comprising:
a photosensor configured to generate photocurrent in response to incident light received from an external scene;
a photocurrent-to-voltage converter coupled to the photosensor to convert the photocurrent to a voltage;
a difference detecting circuit coupled to the photocurrent-to-voltage converter to generate a filtered and amplified signal in response to differences that are detected in the voltage received from the photocurrent-to-voltage converter over time;
event generating comparators coupled to the difference detecting circuit to detect events that occur within the external scene based on a comparison of the filtered and amplified signal with one or more thresholds; and
a time aggregation circuit coupled to the event generating comparators to track an aggregate number of events detected by the event generating comparators over a specified timing window.

2. The event vision pixel of claim 1 , wherein the time aggregation circuit includes at least one counter configured to count the aggregate number of events detected by the event generating comparators over the specified timing window.

3. The event vision pixel of claim 2 , wherein the at least one counter includes:
a first capacitor having a top plate and a grounded bottom plate, wherein the top plate of the first capacitor is selectively coupled to a first reference voltage via a first switch and based on a first reset signal; and
a second capacitor having a top plate and a grounded bottom plate, wherein the top plate of the second capacitor is (a) selectively coupled to the top plate of the first capacitor via a second switch and based on an event signal received from the event generating comparators, (b) selectively coupled to a second reference voltage via a third switch and based on a second reset signal, and (c) selectively coupled to a column bitline (i) via an amplifier and a fourth switch and (ii) based on a readout signal.

4. The event vision pixel of claim 2 , wherein the at least one counter includes:
a first counter configured to track an aggregate number of UP events detected by the event generating comparators over the specified timing window, each of the UP events corresponding to a change in the incident light from darker to brighter; and
a second counter configured to track an aggregate number of DOWN events detected by the event generating comparators over the specified timing window, each of the DOWN events corresponding to a change in the incident light from brighter to darker.

5. The event vision pixel of claim 1 , further comprising readout logic coupled to an output of the event generating comparators, wherein the readout logic includes a latch that is triggered when the event generating comparators detect at least one event over the specified timing window.

6. An event vision sensor, comprising:
a plurality of event vision pixels,
wherein:
each event vision pixel of the plurality is configured to generate event data based at least in part on events indicated in incident light received from an external scene,
each event vision pixel of the plurality includes a compression circuit configured to compress the event data prior to readout of the event data from the event vision pixel.

7. The event vision sensor of claim 6 , wherein:
each compression circuit includes a time aggregation circuit; and
to compress the event data, the time aggregation circuit is configured to track a number of the events detected by the corresponding event vision pixel over a specified timing window.

8. The event vision sensor of claim 7 , wherein each time aggregation circuit includes at least one counter configured to count the number of the events detected by the corresponding event vision pixel over the specified timing window.

9. The event vision sensor of claim 7 , wherein:
each time aggregation circuit includes an up/down counter configured to track a difference between (a) a number of UP events detected by the corresponding event vision pixel over the specified timing window and (b) a number of DOWN events detected by the corresponding event vision pixel over the specified timing window;
each of the UP events corresponds to a change in the incident light from darker to brighter; and
each of the DOWN events corresponds to a change in the incident light from brighter to darker.

10. The event vision sensor of claim 7 , wherein each event vision pixel of the plurality further includes readout logic including at least one latch that is triggered when the corresponding event vision pixel detects at least one event over the specified timing window.

11. The event vision sensor of claim 6 , wherein the event vision sensor is configured to form a pseudo-frame based at least in part on event data readout of one or more event vision pixels of the plurality.

12. The event vision sensor of claim 11 , further comprising an event signal processor configured to execute a compression algorithm on the pseudo-frame.

13. The event vision sensor of claim 12 , wherein the event signal processor is configured to execute a lossy compression algorithm on the pseudo-frame.

14. The event vision sensor of claim 13 , wherein the lossy compression algorithm includes Joint Photographic Experts Group (JPEG) compression, Advanced Video Coding (AVC) compression (also known as H264 compression), High Efficiency Video Coding (HEVC) compression (also known as H265 compression), or any combination thereof.

15. A method of operating an event vision sensor including one or more event vision pixels, the method comprising:
for each individual event vision pixel of the one or more event vision pixels—
detecting events indicated in incident light received from an external scene over a timing window, and
tracking a number of the events detected over the timing window.

16. The method of claim 15 , wherein:
tracking the number of events detected over the timing window includes (a) tracking a number of UP events detected over the timing window and (b) tracking a number of DOWN events detected over the timing window;
each of the UP events corresponds to a change in the incident light from darker to brighter; and
each of the DOWN events corresponding to a change in the incident light from brighter to darker.

17. The method of claim 15 , wherein:
tracking the number of events detected over the timing window includes tracking a difference between (a) a number of UP events detected over the timing window and (b) a number of DOWN events detected over the timing window;
each of the UP events corresponds to a change in the incident light from darker to brighter; and
each of the DOWN events corresponds to a change in the incident light from brighter to darker.

18. The method of claim 15 , further comprising, at or after an end of the timing window, reading out the number of the events detected by at least one of the one or more event vision pixels over the timing window.

19. The method of claim 15 , further comprising, for each of the individual event vision pixels of the one or more event vision pixels, reading out the number of the events detected over the timing window only when the individual event vision pixel detected at least one event during the timing window.

20. The method of claim 15 , further comprising forming a pseudo-frame corresponding to the timing window based at least in part on the numbers of events detected by all or a subset of the one or more event vision pixels over the timing window.

Assignees

• Omnivision Tech Inc

Inventors

• Feng Kaijun
• Suess Andreas

Classifications

• H04N25/77

  Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components · CPC title

• H04N19/172

  the region being a picture, frame or field · CPC title

• H04N19/42

  characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence) · CPC title

• H04N25/47 Primary

  Image sensors with pixel address output; Event-driven image sensors; Selection of pixels to be read out based on image data · CPC title