Automatic surveillance video matting using a shape prior

Patent metadata

Field	Value
Publication number	US-9305357-B2
Application number	US-201113290928-A
Country	US
Kind code	B2
Filing date	Nov 7, 2011
Priority date	Nov 7, 2011
Publication date	Apr 5, 2016
Grant date	Apr 5, 2016

Abstract

A novel technique for performing video matting, which is built upon a proposed image matting algorithm that is fully automatic is disclosed. The disclosed methods utilize a PCA-based shape model as a prior for guiding the matting process, so that manual interactions required by most existing image matting methods are unnecessary. By applying the image matting algorithm to these foreground windows, on a per frame basis, a fully automated video matting process is attainable. The process of aligning the shape model with the object is simultaneously optimized based on a quadratic cost function.

First claim

What is claimed is:

1. A method comprising:
computing an initial alpha matte; and automatically updating the initial alpha matte, wherein automatically updating comprises:
incorporating one or more shape models into the alpha matte;
incorporating one or more alignment parameters into the alpha matte; and
optimizing the alpha matte; and
wherein incorporating the one or more alignment parameters into the alpha matte comprises: calculating an updated alpha matte; calculating a shape basis coefficient; and calculating a transformation parameter.

2. The method of claim 1 , wherein computing the initial alpha matte comprises calculating a Laplacian matting.

3. The method of claim 1 , wherein incorporating the one or more shape models comprises selecting one or more prior shapes from a database.

4. The method of claim 1 , wherein incorporating the one or more shape models comprises incorporating one or more principal component analysis (PCA) based shape models.

5. The method of claim 1 , wherein each of the calculating steps occurs substantially simultaneously.

6. The method of claim 1 , wherein optimizing the alpha matte comprises iteratively repeating the step of incorporating the one or more alignment parameters.

7. The method of claim 6 , wherein iteratively repeating comprises iteratively repeating less than 20 times.

8. The method of claim 1 , wherein calculating the initial alpha matte comprises extracting a foreground image from a background of a still image.

9. The method of claim 1 , wherein calculating the initial alpha matte comprises extracting a foreground image from a background of a video frame.

10. The method of claim 1 , wherein calculating the initial alpha matte comprises calculating a plurality of alpha mattes by extracting a foreground image from a background for each frame of a video comprising a plurality of frames.

11. The method of claim 1 , wherein automatically updating the initial alpha matte does not include marking a foreground region of the alpha matte or a background region of the alpha matte.

12. A method for automatic video matting comprising:
obtaining a plurality of video images;
utilizing a respective one of a plurality of shape priors to estimate an alpha matte for each of the plurality of video images;
aligning each of the video images to estimate the alpha matte for each of the plurality of video images;
optimizing the alpha matte for each of the plurality of video images, wherein optimizing comprises utilizing a quadratic cost function, and simultaneously utilizing the respective one of the plurality of shape priors and aligning each of the video images.

13. The method of claim 12 , wherein each of the plurality of video images is obtained from a person detector.

14. The method of claim 12 , wherein optimizing comprises simultaneously calculating the alpha matte, shape basis coefficients and transformation parameters.

15. The method of claim 12 , comprising iteratively repeating each of the utilizing and aligning steps.

16. A system comprising:
a database having a plurality of shape models stored therein; and
a processor configured to receive a plurality of images and to align a respective one of the plurality of shape models with each of the plurality of images to produce an alpha matte corresponding to each of the plurality of images,
wherein the processor is configured to calculate a quadratic cost function by simultaneously calculating the alpha matte, shape basis coefficients and transformation parameters.

17. The system of claim 16 , comprising a detector configured to detect the plurality of images and transmit the plurality of images to the processor.

18. The system of claim 17 , wherein the detector is a person detector and the plurality of images comprise video images.

19. The system of claim 16 , wherein the processor is configured to align each of the plurality of images with a respective one of the plurality of shape models through a spatial transformation.

Assignees

• Yu Ting
• Tu Peter Henry
• Liu Xiaoming
• Lim Ser Nam
• Gen Electric

Inventors

• Yu Ting
• Tu Peter Henry
• Liu Xiaoming
• Lim Ser Nam

Classifications

• H04N5/272 Primary

  Means for inserting a foreground image in a background image, i.e. inlay, outlay · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/7557

  based on appearance, e.g. active appearance models [AAM] · CPC title

• G06T7/0081 Primary

  Physics · mapped topic

• G06K9/621

  Physics · mapped topic