Fusion based method for in-vehicle violence detection using cameras

Patent metadata

Field	Value
Publication number	US-12525035-B2
Application number	US-202217846548-A
Country	US
Kind code	B2
Filing date	Jun 22, 2022
Priority date	Jun 22, 2022
Publication date	Jan 13, 2026
Grant date	Jan 13, 2026

Abstract

An in-vehicle violence detection and fusion system includes a standard vehicle camera positioned within a vehicle capturing images of at least one occupant of the vehicle. An event camera positioned within the vehicle captures motion images of the at least one occupant. A standard vehicle camera-based region of interest (ROI) signal and an event camera-based ROI signal are received and fused by an ROI fusion module to generate a fused ROI signal. A threat evaluation module receives the fused ROI signal and augments the fused ROI signal with saved data to assist in differentiating normal events from threat events. An event camera-based violence indicator identifies the threat events to the at least one occupant of the vehicle.

First claim

What is claimed is:

1 . A method to detect and fuse in-vehicle violence using cameras, comprising:
providing a standard vehicle camera within a vehicle capturing images of at least one occupant of the vehicle; providing an event camera within the vehicle capturing motion images of the at least one occupant; generating a standard vehicle camera-based region of interest (ROI) signal; generating an event camera-based ROI signal; wherein generating the standard vehicle camera-based ROI signal and generating the event camera ROI signal includes:
dividing the camera images from the standard vehicle camera and the motion images from the event camera into grids differentiated as positive and negative events, the grids having multiple grid cells individually having an event count quantity indicating a quantity of changed events registered in the grid cells representing the camera image over a time interval;
creating a circular buffer to track quantity changes in individual ones of the multiple grid cells over the time interval;
updating the event count quantity for every time interval Δ t , and populating a next grid having multiple grid cells;
moving a pointer to a next one of the grid cells of the next grid, and updating the next one of the grid cells; and
identifying a changing count of the next one of the grid cells using a circular buffer;
applying an ROI fusion module to receive and fuse the event camera-based ROI signal and the standard vehicle camera-based ROI signal and to generate a fused ROI signal; sending the fused ROI signal to a threat evaluation module to augment the fused ROI signal with saved data to assist in differentiating normal events from violent or threat events; and identifying the threat events to the at least one occupant of the vehicle using an event camera-based violence indicator.

2 . The method of claim 1 , further including detecting and tracking body skeletons by receiving images from a privacy masking module and distinguishing human anatomy within the vehicle based on the saved data retrieved from a database in a memory of a computer having saved images of human shapes and movements.

3 . The method of claim 1 , further including detecting violent objects by receiving images from a privacy masking module and distinguishing dangerous activity objects detected within the vehicle.

4 . The method of claim 1 , further including identifying abnormal exposure by receiving an output from the event camera, distinguishing normal light and contrast exposure from high contrast exposure, and generating an abnormal exposure signal if the output from the event camera exceeds a predetermined event camera output threshold.

5 . The method of claim 4 , further including adjusting standard vehicle camera exposure by receiving the abnormal exposure signal and generating and sending exposure change signals to the standard vehicle camera.

6 . The method of claim 1 , wherein:
the event camera has a high speed capturing images in less than 1 ms; and the standard vehicle camera operates at a low speed less than 60 fps.

7 . The method of claim 6 , wherein:
the event camera has a high dynamic range defining a range greater than 140 db; and the standard vehicle camera has a low dynamic range defining a range less than 100 db.

8 . The method of claim 1 , further including identifying a positive event grid by an increasing count of events in the grid cell indicating an increasing quantity of motion events over the time interval.

9 . The method of claim 8 , further including identifying a negative event grid by a decreasing count of events in the grid cell indicating a decreasing quantity of motion events over the time interval.

10 . The method of claim 1 , further including:
identifying when an overall positive (+) event count occurs in a majority of the grid cells of one of the grids indicating an increase in an overall scene brightness; and reducing an exposure for the standard vehicle camera.

11 . The method of claim 1 , further including:
identifying when an overall negative (−) event count occurs in a majority of the grid cells of one of the grids indicating a decrease in an overall scene brightness; and increasing an exposure for the standard vehicle camera.

12 . The method of claim 1 , further including deleting an expired event count if no change in the count has been registered by filling in the event count of one of the grid cells unchanged from a previous event count with a quantity zero (0).

13 . The method of claim 1 , further including consolidating the event camera-based violence indicator by applying contextual information including a vehicle location and a time of day.

14 . The method of claim 1 , further including:
applying a privacy masking to secure privacy of the at least one occupant of the vehicle; and forwarding an output of the standard vehicle camera to a privacy masking module to generate the standard vehicle camera-based region of interest (ROI) signal.

Assignees

• Gm Global Tech Operations Llc

Inventors

• Tong Wei
• Khamis Alaa M
• Aluru Sai Vishnu

Classifications

• H04N23/73

  by influencing the exposure time · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06T7/292

  Multi-camera tracking · CPC title

• G06T7/215

  Motion-based segmentation · CPC title

• G06T2207/30268

  Vehicle interior · CPC title