Ensemble sparse models for image analysis and restoration

Patent metadata

Field	Value
Publication number	US-9875428-B2
Application number	US-201414772343-A
Country	US
Kind code	B2
Filing date	Mar 14, 2014
Priority date	Mar 15, 2013
Publication date	Jan 23, 2018
Grant date	Jan 23, 2018

Abstract

Methods and systems for recovering corrupted/degraded images using approximations obtained from an ensemble of multiple sparse models are disclosed. Sparse models may represent images parsimoniously using elementary patterns from a “dictionary” matrix. Various embodiments of the present disclosure involve simple and computationally efficient dictionary design approach along with low-complexity reconstruction procedure that may use a parallel-friendly table-lookup process. Multiple dictionaries in an ensemble model may be inferred sequentially using greedy forward-selection approach and can incorporate bagging/boosting strategies, taking into account application-specific degradation. Recovery performance obtained using the proposed approaches with image super resolution and compressive recovery can be comparable to or better than existing sparse modeling based approaches, at reduced computational complexity. By including ensemble models in hierarchical multilevel learning, where multiple dictionaries are inferred in each level, further performance improvements can be obtained in image recovery, without significant increase in computational complexity.

First claim

What is claimed is:

1. A method for performing image recovery comprising:
generating, using at least one processor, an ensemble of sparse models from uncorrupted training data, comprising:
learning a dictionary from the uncorrupted training data by randomly selecting a subset of the uncorrupted training data to form the dictionary, and
learning at least one weight from the uncorrupted training data using an unconstrained least squares approximation, the at least one weight corresponding to the dictionary;

using respective sparse models of the ensemble of sparse models to generate a plurality of individual approximations of a first image, wherein each individual approximation of the plurality of individual approximations corresponds to a particular sparse model of the respective sparse models; and
aggregating, using the at least one processor, the individual approximations to generate a second image, by
calculating a linear combination of the plurality of individual approximations with pre-determined weights applied to each individual approximation of the plurality of individual approximations.

2. The method of claim 1 , wherein the first image is a degraded image and the second image is an improved image, wherein the respective sparse models are individual weak sparse models, and wherein aggregating the individual approximations comprises calculating a weighted average of the individual approximations.

3. The method of claim 1 , wherein generating an ensemble of sparse models comprises learning at least one dictionary and at least one weight corresponding to the at least one dictionary from the uncorrupted training data sequentially, using a greedy forward selection procedure.

4. The method of claim 1 , wherein generating an ensemble of sparse models comprises:
degrading uncorrupted training data; and
learning at least one dictionary from the degraded uncorrupted training data.

5. The method of claim 1 , wherein each individual approximation of the individual approximations is determined using a low-complexity correlate-and-maximum procedure and wherein the sparse models represent images parsimoniously using elementary patterns from a dictionary matrix.

6. A system for performing image recovery comprising: at least one processor to:
generate an ensemble of sparse models from uncorrupted training data, comprising
learning a dictionary from the uncorrupted training data by randomly selecting a subset of the uncorrupted training data to form the dictionary; and
learning at least one weight from the uncorrupted training data using an unconstrained least squares approximation, the at least one weight corresponding to the dictionary;

use respective sparse models of the ensemble of sparse models to generate a plurality of individual approximations of a first image, wherein each individual approximation of the plurality of individual approximations corresponds to a particular sparse model of the respective sparse models; and
aggregate the individual approximations to generate a second image,
wherein each individual approximation of the individual approximations is determined using a low-complexity correlate-and-maximum procedure, and wherein the sparse models represent images parsimoniously using elementary patterns from a dictionary matrix.

7. The system of claim 6 , wherein aggregating the individual approximations comprises calculating a linear combination of the plurality of individual approximations with pre-determined weights applied to each individual approximation of the plurality of individual approximations.

8. The system of claim 6 , wherein the first image is a degraded image and the second image is an improved image, wherein the respective sparse models are individual weak sparse models, and wherein aggregating the individual approximations comprises calculating a weighted average of the individual approximations.

9. The system of claim 6 , wherein generating an ensemble of sparse models comprises learning at least one dictionary and at least one weight corresponding to the at least one dictionary from the uncorrupted training data sequentially, using a greedy forward selection procedure.

10. The system of claim 6 , wherein generating an ensemble of sparse models comprises:
degrading uncorrupted training data; and
learning at least one dictionary from the degraded uncorrupted training data.

11. A non-transitory computer-readable medium encoded with instructions for performing image recovery, the instructions executable by a processor, comprising:
generating an ensemble of sparse models from uncorrupted training data, comprising
learning a dictionary from the uncorrupted training data by randomly selecting a subset of the uncorrupted training data to form the dictionary; and learning at least one weight from the uncorrupted training data using an unconstrained least squares approximation, the at least one weight corresponding to the dictionary

using respective sparse models of the ensemble of sparse models to generate a plurality of individual approximations of a first image, wherein each individual approximation of the plurality of individual approximations corresponds to a particular sparse model of the respective sparse models; and
aggregating the individual approximations to generate a second image,
wherein the first image is a degraded image and the second image is an improved image, wherein the respective sparse models are individual weak sparse models, and wherein aggregating the individual approximations comprises calculating a weighted average of the individual approximations.

12. The non-transitory computer-readable medium of claim 11 , wherein aggregating the individual approximations comprises calculating a linear combination of the plurality of individual approximations with pre-determined weights applied to each individual approximation of the plurality of individual approximations.

13. The non-transitory computer-readable medium of claim 11 , wherein generating an ensemble of sparse models comprises:
learning at least one dictionary from the uncorrupted training data by randomly selecting a subset of the uncorrupted training data to form the at least one dictionary; and
learning at least one weight from the uncorrupted training data using an unconstrained least squares approximation, the at least one weight corresponding to the at least one dictionary.

14. The non-transitory computer-readable medium of claim 11 , wherein generating an ensemble of sparse models comprises learning at least one dictionary and at least one weight corresponding to the at least one dictionary from the uncorrupted training data sequentially, using a greedy forward selection procedure.

15. The non-transitory computer-readable medium of claim 11 , wherein each individual approximation of the individual approximations is determined using a low-complexity correlate-and-maximum procedure, and wherein the sparse models represent images parsimoniously using elementary patterns from a dictionary matrix.

Assignees

• Univ Arizona State

Inventors

• Ramamurthy Karthikeyan
• Thiagarajan Jayaraman
• Sattigeri Prasanna
• Spanias Andreas

Classifications

• G06V10/772

  Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries · CPC title

• G06V10/7715 Primary

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06F18/21345

  enforcing sparsity or involving a domain transformation · CPC title

• G06F18/28

  Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title