Determining quality of an image or video using a distortion classifier

Patent metadata

Field	Value
Publication number	US-9595090-B2
Application number	US-201414188413-A
Country	US
Kind code	B2
Filing date	Feb 24, 2014
Priority date	May 10, 2010
Publication date	Mar 14, 2017
Grant date	Mar 14, 2017

Abstract

Techniques and structures are disclosed in which one or more distortion categories are identified for an image or video, and a quality of the image or video is determined based on the one or more distortion categories. The image or video may be of a natural scene, and may be of unknown provenance. Identifying a distortion category and/or determining a quality may be performed without any corresponding reference (e.g., undistorted) image or video. Identifying a distortion category may be performed using a distortion classifier. Quality may be determined with respect to a plurality of human opinion scores that correspond to a particular distortion category to which an image or video of unknown provenance is identified as belonging. Various statistical methods may be used in performing said identifying and said determining, including use of generalized Gaussian distribution density models and natural scene statistics.

First claim

What is claimed is:

1. A method, comprising:
a computer system identifying one or more distortion categories for one or more image frames, wherein the identifying is based on distorted image statistics for the one or more image frames; and
the computer system determining a quality of the one or more image frames based on the identified one or more distortion categories by assessing one or more feature scores corresponding to one or more feature vectors of the one or more image frames, and mapping from a multidimensional space to a quality score, wherein the multidimensional space is defined by ranges for the one or more feature scores;
wherein reference image frames for the one or more image frames are not available to the computer system in performing the identifying and the determining.

2. The method of claim 1 , wherein the one or more image frames are a video.

3. The method of claim 1 , further comprising generating the distorted image statistics for the one or more image frames, wherein generating the distorted image statistics includes applying a wavelet transformation to the one or more image frames.

4. The method of claim 3 , wherein the generating includes applying the wavelet transformation over a plurality of scales and/or a plurality of orientations to produce a plurality of oriented sub-bands.

5. The method of claim 1 , wherein the distorted image statistics for the one or more image frames are generated by using a discrete cosine transformation on the one or more image frames.

6. The method of claim 1 , further comprising rejecting the one or more image frames based on the determined quality of the one or more image frames being below a threshold quality level wherein determining a quality of the one or more image frames includes.

7. The method of claim 1 , wherein the determining includes determining a quality score based on a quality database that includes a plurality of groups of one or more human-generated quality opinion scores, wherein each of the groups of one or more human-generated quality opinion scores corresponds to a respective group of one or more image frames.

8. The method of claim 1 , further comprising training a distortion classifier that is configured to identify distortion categories based on distorted image statistics, wherein the training includes:
applying a given distortion type to a plurality of reference images to produce a plurality of distorted images; and
fitting one or more functions to the plurality of distorted images, wherein the one or more functions are usable to determine a probability that the given distortion type applies to a given distorted image.

9. The method of claim 1 , wherein the identifying and the determining are performed in response to receiving an upload of the one or more image frames from a user.

10. The method of claim 1 , further comprising the computer system performing one or more corrections on the one or more image frames, wherein performing the one or more corrections is in response to determining the quality of the one or more image frames and is based on the identified one or more distortion categories.

11. A computer system, comprising:
a processor; and
a memory having stored thereon instructions that are executable to cause the computer system to perform operations comprising:
identifying one or more distortion categories for one or more image frames, wherein the identifying is based on distorted image statistics for the one or more image frames; and
determining a quality of the one or more image frames based on the identified one or more distortion categories by assessing one or more feature scores corresponding to one or more feature vectors of the one or more image frames, and mapping from a multidimensional space to a quality score, wherein the multidimensional space is defined by ranges for the one or more feature scores;
wherein the identifying and the determining do not use reference image frames for the one or more image frames.

12. The computer system of claim 11 , wherein the operations further comprise:
operating as a web server; and
receiving the one or more image frames from a user device via a submission page transmitted by the web server to the user device.

13. The computer system of claim 11 , wherein the operations further comprise performing one or more corrections on the one or more image frames, wherein performing the one or more corrections is in response to determining the quality of the one or more image frames, and is based on the identified one or more distortion categories.

14. The computer system of claim 11 , wherein determining the quality of the one or more image frames includes determining a quality score based on a quality database that includes a plurality of groups of one or more human-generated quality opinion scores, wherein each of the groups of one or more human-generated quality opinion scores corresponds to a respective group of one or more image frames.

15. The computer system of claim 11 , wherein the operations further comprise generating the distorted image statistics for the one or more image frames, wherein generating the distorted image statistics includes applying a wavelet transformation to the one or more image frames.

16. A non-transitory computer readable storage medium having instructions stored thereon that are executable to cause a computer system to perform operations comprising:
training a distortion classifier that is configured to identify distortion categories based on distorted image statistics, wherein the training includes:
applying a given distortion type to a plurality of reference images to produce a plurality of distorted images; and
fitting one or more functions to the plurality of distorted images, wherein the one or more functions are usable to determine a probability that the given distortion type applies to a given distorted image;

receiving one or more image frames, but not reference image frames for the one or more image frames;
identifying, via the trained distortion classifier, one or more distortion categories for the one or more image frames; and
determining a quality of the one or more image frames based on the identified one or more distortion categories.

17. The non-transitory computer readable storage medium of claim 16 , wherein the operations further comprise applying one or more corrections to the one or more image frames based on the identified one or more distortion categories.

18. The non-transitory computer readable storage medium of claim 16 , wherein the operations further comprise receiving the one or more image frames from a user device via the internet.

19. The non-transitory computer readable storage medium of claim 16 , wherein the one or more image frames are taken from a video.

20. The non-transitory computer readable storage medium of claim 16 , wherein determining a quality of the one or more image frames includes:
assessing one or more feature scores corresponding to one or more feature vectors of the one or more image frames; and
mapping from a multidimensional space to a quality score, wherein the multidimensional space is defined by ranges for the one or more feature scores.

Assignees

• Univ Texas

Inventors

• Bovik Alan
• Moorthy Anush

Classifications

• G06V10/993

  Evaluation of the quality of the acquired pattern · CPC title

• G06K9/00711

  Physics · mapped topic

• G06K9/00664

  Physics · mapped topic

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/30168

  Image quality inspection · CPC title