Surveillance video based video foreground extraction method

Patent metadata

Field	Value
Publication number	US-10395374-B2
Application number	US-201716089517-A
Country	US
Kind code	B2
Filing date	Apr 6, 2017
Priority date	Jun 27, 2016
Publication date	Aug 27, 2019
Grant date	Aug 27, 2019

Abstract

Disclosed in the present invention is a video foreground extraction method for a surveillance video, which adjusts a size of a block to adapt to different video resolutions based on an image block processing method; and then extracts a foreground object in a moving state by establishing a background block model, the method comprising: representing each frame of image I in the surveillance video as a block; initializing; updating a block background weight, a block temporary background and a temporary background; updating a block background and a background; saving a foreground, and updating a foreground block weight and a foreground block; and performing binarization processing on the foreground to obtain a final foreground result. The invention reduces the dimension of each frame of image in the surveillance video, eliminates the influence of the illumination on the block method by means of the pixel-based comparison, and meanwhile eliminates the influence on video analysis due to increasing size of the surveillance video. The invention can extract a dynamic foreground object, adapt to a larger size of a surveillance video, and has a lower algorithm complexity.

First claim

The invention claimed is:

1. A surveillance video based video foreground extraction method which adjusts a size of a block to adapt to different video resolutions extracts a foreground object in a moving state, the method comprising the following steps:
step 1: representing each frame of an image I in the surveillance video as the block and representing the image I as a matrix block I′;
step 2: initializing for a first frame of the image I a temporary background β and a background B with the image I, wherein a temporary foreground is represented with a zero matrix γ with the size of P*Q, a foreground is represented with a matrix F and initialized with the zero matrix, a block background A and a temporary block background α are initialized with a matrix I′, a background block weight is represented with a zero matrix ψ with the size of m*n, a foreground block weight is represented with φ and initialized with the zero matrix, and a foreground block end is represented with ω and initialized with the zero matrix;
step 3: setting an s and a t to represent respective rows and columns of a matrix block, comparing an element b s,t I′ i in a matrix block I′ i in a next frame of the image I with a corresponding element b s,t I′ i-1 in a previous frame of the image I to obtain a difference |b s,t I′ i −b s,t I′ i-1 |, and setting a threshold ε 1 , wherein if |b s,t I′ i −b s,t I′ i-1 |<ε 1 , a block background weight ψ s,t is auto-increased by 1; otherwise the block background weight ψ s,t is reset to be 0, and a block temporary background and the temporary background are updated;
step 4: representing a duration that a block stream remains unchanged with a set threshold ε 2 , wherein if the value of a block background image weight exceeds the threshold ε 2 , the block background is updated with the temporary block background, and the background is updated with the temporary background;
step 5: when the difference between a current block and a background block exceeds the threshold ε 1 or the background block weight is 0, executing step 6; otherwise, executing step 3;
step 6: comparing pixels in the current block with pixels in a block corresponding to a background frame, wherein if a pixel difference exceeds a number of pixels of a threshold ε 3 , that is, half of the number of pixels in the block, executing step 7; otherwise, executing step 3;
step 7: saving the difference between a pixel value in the image I and the pixel value corresponding to the background B as a foreground F, wherein if the foreground block is continuous and the difference between current and previous foreground blocks is less than the threshold ε 1 , the foreground block weight is auto-increased by 1 and the foreground blocks are weighted and averaged;
step 8: if the value of a block foreground image weight exceeds the threshold ε 2 , updating the background B, the block background A, and the temporary block background α with the temporary foreground, resetting the block background weight to be the threshold ε 2 , and setting the temporary foreground γ as 0;
step 9: binarizing the foreground F obtained in step 7 to obtain a final foreground result, and saving a current frame number as a foreground end ω s,t ; and
step 10: repeating steps 3-9 until all frames of the image I in the video are processed so as to obtain a video foreground through extraction.

2. The surveillance video based video foreground extraction method according to claim 1 , wherein the step 1 of representing the image I as the matrix block image I′ further comprises:
firstly graying the image I and calculating an integral image, and then calculating a sum of pixels in the blocks of different sizes via the integral image, and obtaining an image matrix block image I′ with a smaller dimension instead of the image I, wherein the sum of pixels in the block represents all pixels in the block.

3. The surveillance video based video foreground extraction method according to claim 1 , wherein the size of the block is set to be p*q, ε 1 is a threshold representing the difference between blocks, the threshold ε 3 is the threshold of the difference between pixels, and

ɛ
1

=

p
*
q
*

ɛ
3

2

is set.

4. The surveillance video based video foreground extraction method according to claim 1 , wherein the threshold ε 2 is 15.

5. The surveillance video based video foreground extraction method according to claim 1 , wherein the threshold ε 3 is 30.

6. The surveillance video based video foreground extraction method according to claim 1 , wherein the step 8 of updating the background B, the block background A and the temporary block background α with the temporary foreground is specifically calculated via the following equation:
s,t =γ s,t , s,t =α s,t =Σp ( p∈γ s,t ), ψ s,t =φ s,t φ s,t >ε 2

wherein B is the background; s and t represent rows and columns of a corresponding new matrix respectively; γ represents the temporary foreground; A represents the block background; α represents the temporary block background; ψ represents the background block weight; and φ represents the foreground block weight.

Assignees

• Univ Peking Shenzhen Graduate School

Inventors

• Li Ge
• Zang Xianghao
• Wang Wenmin
• Wang Ronggang

Classifications

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06T7/136

  involving thresholding · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T7/194 Primary

  involving foreground-background segmentation · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title