Unsupervised image-based anomaly detection using multi-scale context-dependent deep autoencoding gaussian mixture model

Patent metadata

Field	Value
Publication number	US-10853937-B2
Application number	US-201916248955-A
Country	US
Kind code	B2
Filing date	Jan 16, 2019
Priority date	Feb 5, 2018
Publication date	Dec 1, 2020
Grant date	Dec 1, 2020

Abstract

A false alarm reduction system is provided that includes a processor cropping each input image at randomly chosen positions to form cropped images of a same size at different scales in different contexts. The system further includes a CONDA-GMM, having a first and a second conditional deep autoencoder for respectively (i) taking each cropped image without a respective center block as input for measuring a discrepancy between a reconstructed and a target center block, and (ii) taking an entirety of cropped images with the target center block. The CONDA-GMM constructs density estimates based on reconstruction error features and low-dimensional embedding representations derived from image encodings. The processor determines an anomaly existence based on a prediction of a likelihood of the anomaly existing in a framework of a CGMM, given the context being a representation of the cropped image with the center block removed and having a discrepancy above a threshold.

First claim

What is claimed is:

1. A false alarm reduction system for reducing false alarms in an automatic defect detection system, the false alarm reduction system comprising:
a processor for performing a cropping operation on each of input images at randomly chosen positions to form a set of cropped images of a same size at different scales in different contexts; and
a CONtext-conditional Deep Autoencoding Gaussian Mixture Model (CONDA-GMM), having a first and a second conditional deep autoencoder for respectively (i) taking each of the cropped images without a respective center block as input for measuring a discrepancy between a reconstructed center block and the target center block, and (ii) taking an entirety of the cropped images with the target center block, the CONDA-GMM constructing density estimates based on both reconstruction error features and low-dimensional embedding representations derived from encodings of the cropped images,
wherein the processor determines an existence of an anomaly based on a prediction of a likelihood of the anomaly existing in a framework of a context-dependent Conditional Gaussian Mixture Model (CGMM), given the context being a representation of the cropped image with the center block removed and having a discrepancy above a threshold amount.

2. The false alarm reduction system of claim 1 , wherein the CONDA-GMM comprises a context-dependent deep autoencoding compression network and a context-dependent density estimation network.

3. The false alarm reduction system of claim 1 , wherein each of the input images is cropped at multiple different randomly chosen positions.

4. The false alarm reduction system of claim 1 , wherein a cropping-specific contextual vector is formed using image features of the cropped images.

5. The false alarm reduction system of claim 4 , wherein the image features comprise Scale Invariant Feature Transformation (SIFT) features.

6. The false alarm reduction system of claim 4 , wherein the image features forming the cropping-specific contextual vector comprise edges of the cropped images, wherein the cropped edges have been cropped to be without a center region.

7. The false alarm reduction system of claim 1 , wherein the CONDA-GMM includes a first and a second conditional deep autoencoder for respectively taking each of the cropped images without a respective center block as input for measuring a discrepancy between a reconstructed center block and the target center block, and taking an entirety of the cropped images with the target center block.

8. The false alarm reduction system of claim 1 , wherein the likelihood is based on an energy level.

9. The false alarm reduction system of claim 1 , wherein both reconstruction errors and log-likelihood are used for the anomaly detection.

10. The false alarm reduction system of claim 1 , further comprising a Multi-Layer Perceptron, predicting an input-specific mixture coefficient conditioned on the representation.

11. A false alarm reduction method for reducing false alarms in an automatic defect detection system, the false alarm reduction method comprising:
performing, by a processor, a cropping operation on each of input images at randomly chosen positions to form a set of cropped images of a same size at different scales in different contexts;
encoding, by a first conditional deep autoencoder of a CONtext-conditional Deep Autoencoding Gaussian Mixture Model (CONDA-GMM), each of the cropped images without a respective center block as input for measuring a discrepancy between a reconstructed center block and the target center block, and
encoding, by a second conditional deep autoencoder of the CONDA-GMM, an entirety of the cropped images with the target center block;
constructing, by the CONDA-GMM, density estimates based on both reconstruction error features and low-dimensional embedding representations derived from encodings of the cropped images; and
determining, by the processor, an existence of an anomaly based on a prediction of a likelihood of the anomaly existing in a framework of a context-dependent Conditional Gaussian Mixture Model (CGMM), given the context being a representation of the cropped image with the center block removed and having a discrepancy above a threshold amount.

12. The false alarm reduction method of claim 11 , wherein the CONDA-GMM comprises a context-dependent deep autoencoding compression network and a context-dependent density estimation network.

13. The false alarm reduction method of claim 11 , wherein each of the input images is cropped at multiple different randomly chosen positions.

14. The false alarm reduction method of claim 11 , further comprising forming a cropping-specific contextual vector using image features of the cropped images.

15. The false alarm reduction method of claim 14 , wherein the image features comprise Scale Invariant Feature Transformation (SIFT) features.

16. The false alarm reduction method of claim 14 , wherein the image features forming the cropping-specific contextual vector comprise edges of the cropped images, wherein the cropped edges have been cropped to be without a center region.

17. The false alarm reduction method of claim 11 , wherein the CONDA-GMM includes a first and a second conditional deep autoencoder for respectively taking each of the cropped images without a respective center block as input for measuring a discrepancy between a reconstructed center block and the target center block, and taking an entirety of the cropped images with the target center block.

18. The false alarm reduction method of claim 11 , wherein the likelihood is based on an energy level.

19. The false alarm reduction method of claim 11 , wherein both reconstruction errors and log-likelihood are used for the anomaly detection.

20. The false alarm reduction method of claim 11 , further comprising predicting, by a Multi-Layer Perceptron, an input-specific mixture coefficient conditioned on the representation.

Assignees

• Nec Lab America Inc
• Nec Corp

Inventors

• Niculescu-Mizil Alexandru
• Min Renqiang
• Cosatto Eric
• Lai Farley
• Graf Hans Peter
• Fontaine Xavier

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06T7/001 Primary

  using an image reference approach · CPC title

• G06T7/0004 Primary

  Industrial image inspection · CPC title

• G06F18/2433

  Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection · CPC title