Feature- and classifier-based vehicle headlight/shadow removal in video

Patent metadata

Field	Value
Publication number	US-9275289-B2
Application number	US-201414227035-A
Country	US
Kind code	B2
Filing date	Mar 27, 2014
Priority date	Mar 27, 2014
Publication date	Mar 1, 2016
Grant date	Mar 1, 2016

Abstract

A method for removing false foreground image content in a foreground detection process performed on a video sequence includes, for each current frame, comparing a feature value of each current pixel against a feature value of a corresponding pixel in a background model. The each current pixel is classified as belonging to one of a candidate foreground image and a background based on the comparing. A first classification image representing the candidate foreground image is generated using the current pixels classified as belonging to the candidate foreground image. The each pixel in the first classification image is classified as belonging to one of a foreground image and a false foreground image using a previously trained classifier. A modified classification image is generated for representing the foreground image using the pixels classified as belonging to the foreground image while the pixels classified as belonging to the false foreground image are removed.

First claim

What is claimed is:

1. A method for removing false foreground image content in a foreground detection process performed on a video sequence, the method comprising:
receiving video data from a sequence of frames taken from an associated image capture device monitoring a scene of interest;
for a current frame, comparing a current feature value of each current pixel against a feature value of a corresponding pixel in a background model of the scene of interest;
classifying the each current pixel as belonging to one of a candidate foreground image and a background based on the comparing;
generating a first classification image representing the candidate foreground image using current pixels classified as belonging to the candidate foreground image;
applying the first classification image to a previously trained classifier in a block-wise manner to generate a binary decision for each pixel-block of the first classification image;
generating a coarse scale classification image based on the binary decision of the each pixel block;
generating a modified classification image representing the foreground image by removing redundant pixels between the coarse scale classification image and the first classification image.

2. The method of claim 1 , wherein the classifying the each pixel as belonging to the one of the foreground image and background includes:
performing a comparison of the current feature value in the current frame relative to the corresponding feature value in the background model.

3. The method of claim 1 , wherein the false foreground detection includes at least one of a shadow and a light cast from a moving object.

4. The method of claim 1 , wherein the applying the first classification image to a previously trained classifier in a block-wise manner to generate a binary decision for each pixel-block includes:
segmenting the first classification image into pixel blocks;
for each pixel block, calculating a first feature vector for the current frame;
calculating a second feature vector for a current background estimate;
comparing the second feature vector with the first feature vector to obtain a difference feature vector corresponding to the pixel block;
inputting the resulting difference feature vector to the previously trained classifier; and,
outputting a classification decision for the each pixel block.

5. The method of claim 1 , further comprising:
computing a convex hull on the modified classification image to generate the foreground image.

6. The method of claim 1 , further comprising training the classifier before receiving the video data, the training including:
locating an object in a sample frame;
computing a feature of at least one segmented block in the sample frame corresponding to the object;
computing a difference between the current feature value and a corresponding feature value for a corresponding one of the at least one segmented block in a background estimate;
labeling each difference as corresponding to one of a foreground object and a false foreground object; and,
training the classifier using the labeled samples and corresponding difference features.

7. The method of claim 6 , wherein the current feature value represents one of a texture and color.

8. The method of claim 6 , wherein the current feature value represents a feature that is invariant to monotonic transformations of color intensities.

9. A system for removing false foreground image content in a video sequence, the system comprising a false foreground image detection device including a memory and a processor in communication with the processor configured to:
receive video data from a sequence of frames taken from an associated image capture device monitoring a scene of interest;
for a current frame, compare a current feature value of each current pixel against a feature value of a corresponding pixel in a background model of the scene of interest;
classify the each current pixel as belonging to one of a candidate foreground image and a background based on the comparing;
generate a first classification image representing the candidate foreground image using current pixels classified as belonging to the candidate foreground image;
apply the first classification image to a previously trained classifier in a block-wise manner to generate a binary decision for each pixel-block of the first classification image;
generate a coarse scale classification image based on the binary decision of the each pixel block; and,
generate a modified classification image representing the foreground image by removing redundant pixels between the coarse scale classification image and the first classification image.

10. The system of claim 9 , wherein the processor is further configured to:
perform a comparison of the current feature value in the current frame relative to a corresponding feature value in the background model for the classifying of the each pixel as belonging to the one of the foreground image and background.

11. The system of claim 9 , wherein the false foreground detection includes at least one of a shadow and a light cast from a moving object.

12. The system of claim 9 , wherein the processor is further configured to:
segment the first classification image into pixel blocks;
for each pixel block, calculate a first feature vector for the current frame;
calculate a second feature vector for a current background estimate;
compare the second feature vector with the first feature vector to obtain a difference feature vector corresponding to the pixel block;
input the resulting difference feature vector to the previously trained classifier; and,
output a classification decision for the each pixel block.

13. The system of claim 9 , wherein the processor is further configured to:
compute a convex hull on the modified classification image to generate the foreground image.

14. The system of claim 9 , wherein the processor is further configured to:
locate an object in a sample frame;
compute a feature of at least one segmented block in the sample frame corresponding to the object;
compute a difference between the current feature value and a corresponding feature value for a same block in a background estimate;
label each difference as corresponding to one of a foreground object and a false foreground object; and,
train a classifier using the labeled samples and corresponding difference features.

15. The system of claim 14 , wherein the current feature value represents one of a texture and color.

16. The system of claim 14 , wherein the current feature value represents a feature that is invariant to monotonic transformations of color intensities.

Assignees

• Xerox Corp

Inventors

• Li Qun
• Bernal Edgar A
• Shreve Matthew
• Loce Robert P

Classifications

• G06T7/215 Primary

  Motion-based segmentation · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/584

  of vehicle lights or traffic lights · CPC title

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title

• G06K9/00825 Primary

  Physics · mapped topic