Method and system for detecting abnormal traffic behavior

Patent metadata

Field	Value
Publication number	US-12555384-B2
Application number	US-202318681766-A
Country	US
Kind code	B2
Filing date	Apr 20, 2023
Priority date	Apr 28, 2022
Publication date	Feb 17, 2026
Grant date	Feb 17, 2026

Abstract

The present disclosure discloses a method and a system for detecting an abnormal traffic behavior. The method of the present disclosure includes: retaining an abnormal static target vehicle in a traffic surveillance video in a background through background modeling; performing abnormal target detection, and obtaining a cropped picture of an abnormal target vehicle and a cropped video clip through cropping; performing anomaly start time estimation, inputting the cropped picture and the cropped video clip to a network model combining twin cross-correlation with pseudo three-dimensional (P3D)-Attention, labeling a classification label on the cropped video clip, and determining a video frame when abnormal behavior occurs; and determining whether a to-be-matched vehicle is an abnormal target vehicle, and determining a start time and an end time of abnormal traffic behavior with reference to the video frame that is obtained when the abnormal behavior occurs.

First claim

What is claimed is:

1 . A method for detecting an abnormal traffic behavior, comprising:
performing background modeling and removing a vehicle that moves normally in each frame of a traffic surveillance video from a framework, to enable an abnormal static target vehicle to be retained in a background; performing perspective view cropping on each frame of background extracted through background modeling, and obtaining, according to a vehicle size, a cropping box used for cropping the traffic surveillance video; performing abnormal target detection on the traffic surveillance video once every first quantity of frames, to detect abnormal target vehicles in a video frame of the traffic surveillance video; and after an abnormal target vehicle is detected, obtaining a cropped picture of the abnormal target vehicle through cropping, and forward or backward cropping the traffic surveillance video, by using a location of the abnormal target vehicle as a center, to obtain a cropped video clip whose space capacity is several times the vehicle size and whose time is a second quantity of frames, for subsequent anomaly time estimation; and performing anomaly start time estimation, abnormal vehicle status detection and abnormal target matching according to a detection result of the abnormal target vehicle, wherein the abnormal vehicle status detection is to input the cropped picture of the abnormal target vehicle and the cropped video clip to a network model combining twin cross-correlation with pseudo three-dimensional (P3D)-Attention, detect whether an abnormal target in the cropped video clip is in a static state or a driving state, separately label a classification label on the cropped video clip according to a result of the abnormal vehicle status detection, separately mark the cropped video clip as anomaly, driving, or normal, and determine a video frame when abnormal behavior of the abnormal target vehicle occurs; and the abnormal target matching is to input a to-be-matched vehicle picture and the cropped picture of the abnormal target vehicle to the network model combining twin cross-correlation with P3D-Attention, determine whether the to-be-matched vehicle is an abnormal target vehicle, and determine a start time and an end time of abnormal traffic behavior with reference to the video frame that is obtained when the abnormal behavior of the abnormal target vehicle occurs and that is determined in the detection result of the abnormal target vehicle.

2 . The method for detecting an abnormal traffic behavior according to claim 1 , wherein the background modeling uses a mixture of Gaussians (MOG2) algorithm.

3 . The method for detecting an abnormal traffic behavior according to claim 1 , wherein the abnormal target detection uses a You Only Look Once version 3 (YOLOv3) target detection method.

4 . The method for detecting an abnormal traffic behavior according to claim 1 , wherein inputs of the abnormal vehicle status detection are the cropped picture of the abnormal target vehicle, and a video clip whose space capacity is twice the vehicle size and whose time is the set second quantity of frames and that is obtained by cropping by using the abnormal target vehicle as the center; and
the abnormal vehicle status detection comprises: separately extracting a feature map of the input cropped picture of the abnormal target vehicle and a feature map of each frame of the input video clip by using a P3D-Attention module, and improving correlation of important channel features by using P3D-Attention; then fusing the separately extracted feature maps on three selected layers of different receptive field sizes by using twin cross-correlation operations; for results of first two twin cross-correlation operations, fusing, by using a multiply method, feature maps extracted from the input cropped picture and the input video clip after the twin cross-correlation operations with the feature map of each frame of the input video clip extracted by using the P3D-Attention module before cross-correlation; performing pooling by using global average pooling (GAP); and finally directly classifying the input cropped picture of the abnormal target vehicle and the input video clip by using a softmax layer, separately labeling a classification label on the input video clip according to the result of the abnormal vehicle status detection, separately marking the input video clip as anomaly, driving, or normal, and determining a video frame in which an anomaly time point is located.

5 . The method for detecting an abnormal traffic behavior according to claim 4 , wherein inputs of the abnormal target vehicle matching are the cropped picture of the abnormal target vehicle and a positive/negative-type picture used for an abnormal target matching model; and
the abnormal target vehicle matching comprises: separately extracting feature maps of two input pictures by using the P3D-Attention module, and performing a twin cross-correlation operation by using the extracted feature maps of the two inputs; fusing the separately extracted feature maps on three selected layers of different receptive field sizes by using twin cross-correlation operations; obtaining three feature maps of different sizes, and performing a concatenation operation; and finally directly classifying, by using the softmax layer, the input cropped picture of the abnormal target vehicle and the positive/negative-type picture used for the abnormal target matching model, wherein a classification result is matching or mismatching.

6 . The method for detecting an abnormal traffic behavior according to claim 4 , wherein the P3D-Attention module simulates a 3×3×3 convolution in a spatial domain and a time domain respectively by using a 1×3×3 convolution kernel and a 3×1×1 convolution kernel, and decouples the 3×3×3 convolution in time and space; and the P3D-Attention module further comprises a dual-channel attention model (DCAM) and a spatial attention module (SAM) that improve correlation of important features.

7 . The method for detecting an abnormal traffic behavior according to claim 6 , wherein the DCAM combines the 1×3×3 convolution kernel in the spatial domain with the 3×1×1 convolution kernel in the time domain to form the 3×3×3 convolution, learns a weight of a frame attention module M F ∈R (F×1×1×1) to express attention to a frame, and learns a weight of a channel attention module (CAM) M C ∈R (1×1×1×C) to express attention to a channel; and F represents a quantity of frames of a feature map F∈R (F×H×W×C) , C represents a quantity of channels of the feature map, H represents a height of the feature map, and W represents a width of the feature map.

8 . The method for detecting an abnormal traffic behavior according to claim 7 , wherein the SAM learns location information in a weight matrix M S ∈R (1×W×H×1) of a single-channel feature map by using a two-dimensional (2D) convolution kernel, to determine importance and correlation of each location in a video feature map; and F represents the quantity of frames of the feature map F∈R (F×H×W×C) , C represents the quantity of channels of the feature map, H represents the height of the feature map, and W represents the width of the feature map.

9 . A system for detecting an abnormal traffic behavior, implementing the method for detecting an abnormal traffic behavior according to claim 1 , comprising:
a video acquisition and cropping module, configured to acquire a real-time traffic video stream, provide continuous traffic video stream information, display the information on a display module, and transmit the information as input information to a background modeling module; the background modeling module, used for a reserved interface and configured to perform backg

Assignees

• Univ Nanjing Sci & Tech

Inventors

• Qi Yong
• Zeng Xin
• Zhuang Yuan

Classifications

• G08G1/017

  identifying vehicles (G08G1/015, G08G1/054 take precedence) · CPC title

• G06V10/75

  Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries · CPC title

• G06V20/70 Primary

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title

• G06V10/806

  of extracted features · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title