Camera systems and event-assisted image processing methods

Patent metadata

Field	Value
Publication number	US-12501124-B2
Application number	US-202318396318-A
Country	US
Kind code	B2
Filing date	Dec 26, 2023
Priority date	Dec 26, 2023
Publication date	Dec 16, 2025
Grant date	Dec 16, 2025

Abstract

The present disclosure provides a camera system and an event-assisted image processing method. The camera system includes an image sensor, an event-based sensor, and a processing unit. The image sensor is configured to capture visual images of a targeted scene to obtain image sensing frames with a first frequency. The event-based sensor is configured to capture event data of the targeted scene to obtain event frames with a second frequency higher than the first frequency. The processing unit is configured to: receiving the image sensing frames within a predetermined time period; accumulating the event frames within the predetermined time period; generating a temporal-spatial mask indicating interested areas for the event frames; determining geometric features in the temporal-spatial masks; synchronizing the image sensing frames and the event frames at timestamps within the predetermined time period; and fusing the temporal-spatial mask with the image sensing frames to obtain a masked visual image.

First claim

What is claimed is:

1 . A camera system, comprising:
an image sensor, configured to capture visual images of a targeted scene to obtain image sensing frames with a first frequency; an event-based sensor, configured to capture event data of the targeted scene to obtain event frames with a second frequency higher than the first frequency; a processing unit configured to:
receiving the image sensing frames within a predetermined time period;
accumulating the event frames within the predetermined time period;
generating a temporal-spatial mask indicating interested areas for the event frames;
determining geometric features in the temporal-spatial masks;
synchronizing the image sensing frames and the event frames at timestamps within the predetermined time period; and
fusing the temporal-spatial mask with the image sensing frames to obtain a masked visual image.

2 . The camera system of claim 1 , wherein generating the temporal-spatial mask comprises:
determining whether a scene change occurs in the targeted scene according to the event frames.

3 . The camera system of claim 1 , wherein the event frames indicate light intensity changes occurring in the targeted scene.

4 . The camera system of claim 1 , wherein the temporal-spatial mask is generated under at least one predetermined granularity.

5 . The camera system of claim 1 , further comprising a memory storing geometric shapes of specified objects,
wherein the processing unit is further configured to obtain the geometric shapes from the memory to determine the geometric features in the temporal-spatial masks.

6 . The camera system of claim 5 , wherein the predetermined shapes comprise at least one of points, straight lines, curves, circles, rectangles, triangles, ellipse, and trapezoid.

7 . The camera system of claim 5 , wherein the processing unit is further configured to encode the geometric features into a set of geometric shape parameters.

8 . The camera system of claim 7 , wherein the set of geometric shape parameters is encoded according to a shape formula corresponding to the predetermined shapes.

9 . The camera system of claim 7 , wherein the processing unit is further configured to filter out unreliable shapes.

10 . The camera system of claim 1 , wherein the image sensor is a CMOS image sensor (CIS).

11 . The camera system of claim 1 , wherein a frequency of the timestamps is substantially related to the first frequency.

12 . The camera system of claim 1 , wherein the processing unit is further configured to denoise the event frames.

13 . The camera system of claim 1 , the processing unit is further configured to analyze connected-component for the event frames.

14 . The camera system of claim 13 , wherein the processing unit is further configured to segment the event frames to into granularities.

15 . The camera system of claim 1 , the processing unit is further configured to track movements of specified objects.

16 . An event-assisted image processing method, comprising:
obtaining image sensing frames of a targeted scene with a first frequency within a predetermined time period; obtaining event frames of the targeted scene with a second frequency higher than the first frequency within the predetermined time period; generating a temporal-spatial mask indicating interested areas for the event frames; determining geometric features in the temporal-spatial masks; synchronizing the image sensing frames and the event frames at timestamps within the predetermined time period; and fusing the temporal-spatial mask with the image sensing frames to obtain a masked visual image.

17 . The method of claim 16 , further comprising determining whether a scene change occurs in the targeted scene according to the event frames.

18 . The method of claim 16 , wherein the event frames indicate light intensity changes occurring in the targeted scene.

19 . The method of claim 16 , wherein the determining the geometric features in the temporal-spatial masks is based on stored geometric shapes of specified objects.

20 . The method of claim 19 , further comprising:
determining geometric shapes for the temporal-spatial masks based on a shape-matching algorithm; and encoding the geometric features into a set of geometric shape parameters.

Assignees

• Omnivision Tech Inc

Inventors

• Mu Bo
• Jiang Rui

Classifications

• G06T7/187

  involving region growing; involving region merging; involving connected component labelling · CPC title

• G06T2207/20221

  Image fusion; Image merging · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title