No-reference image and video quality evaluation

Patent metadata

Field	Value
Publication number	US-9706111-B2
Application number	US-201514794260-A
Country	US
Kind code	B2
Filing date	Jul 8, 2015
Priority date	Jul 8, 2015
Publication date	Jul 11, 2017
Grant date	Jul 11, 2017

Abstract

Techniques related to no-reference image and video quality evaluation are discussed. Such techniques may include generating, for a still image or video frame, features including a natural scene statistics based feature and an image quality based feature and determining an image evaluation indicator associated with the still image or video frame based on a mapping of the generated features to the image evaluation indicator.

First claim

What is claimed is:

1. A method for performing no-reference image or video evaluation comprising:
determining a candidate image is a valid image for quality evaluation based at least in part on a composition check of the candidate image or a category check of the candidate image;
generating features associated with the valid image, wherein the features comprise at least one natural scene statistics based feature and at least one image quality based feature and generating the features comprises:
transforming patches of the valid image to generate a plurality of sets of transform coefficients;
fitting a probability distribution to each of a plurality of histograms, wherein each of the plurality of histograms is associated with a position within the sets of transform coefficients, to generate a plurality of probability distribution parameters;
partitioning the probability distribution parameters based on orientation bands and frequency bands associated with the sets of transform coefficients; and
generating at least a first feature based on probability distribution parameters from a first band of the orientation bands and frequency bands; and

determining an image evaluation indicator associated with the valid image based on a mapping of the generated features to the image evaluation indicator.

2. The method of claim 1 , wherein determining the candidate image is a valid image comprises:
determining a probability the candidate image is associated with a predetermined target category based on natural scene statistics associated with the candidate image; and
performing the category check by comparing the probability to a threshold.

3. The method of claim 2 , wherein the predetermined target category comprises at least one of a landscape category, an indoor arrangement category, an indoor flat surface category, or an outdoor night capture category.

4. The method of claim 1 , wherein determining the candidate image is a valid image comprises:
applying a flat region detector to the candidate image to determine a percentage of the candidate image including flat regions; and
performing the composition check by comparing the percentage to a threshold.

5. The method of claim 1 , further comprising:
presenting, to a user via a display, one or more image capture indicators for capturing the candidate image, wherein the image capture indicators comprise at least one of a scene composition indicator, an illumination indicator, or an example image.

6. The method of claim 1 , further comprising:
determining a camera evaluation indicator associated with a camera used to capture the valid image based on a plurality of image evaluation indicators comprising the image evaluation indicator.

7. The method of claim 6 , wherein a portion of the plurality of image evaluation indicators comprises image evaluation indicators that are each associated with an image category of a plurality of image categories.

8. The method of claim 1 , wherein the sets of transform coefficients comprise discrete cosine transform (DCT) coefficients, the probability distribution comprises a generalized Gaussian distribution (GDD), and the first feature comprises a mean of the probability distribution parameters from the first band.

9. The method of claim 1 , further comprising:
generating a second feature as a ratio of a mean of the probability distribution parameters from the first band and a mean of probability distribution parameters from a second band of the orientation bands and frequency bands.

10. The method of claim 1 , wherein the probability distribution comprises a generalized Gaussian distributions (GDD) and the plurality of probability distribution parameters comprise GDD shape parameters, GDD variances, GDD means of absolute values, and GDD standard deviations.

11. The method of claim 1 , wherein the image quality based feature comprises at least one of a sharpness, a noise, a dynamic range, or an illumination of the valid image.

12. The method of claim 1 , wherein the candidate image comprises at least one of a still image or a video frame and the mapping comprises a dynamically trained mapping.

13. A system for performing no-reference image or video evaluation comprising:
a memory configured to receive a valid image for quality evaluation; and
a central processor coupled to the memory, the central processor to:
determine a candidate image is the valid image for quality evaluation based at least in part on a composition check of the candidate image or a category check of the candidate image;
generate features associated with the valid image, wherein the features comprise at least one natural scene statistics based feature and at least one image quality based feature and to generate the features comprises the central processor to:
transform patches of the valid image to generate a plurality of sets of transform coefficients;
fit a probability distribution to each of a plurality of histograms, wherein each of the plurality of histograms is associated with a position within the sets of transform coefficients, to generate a plurality of probability distribution parameters;
partition the probability distribution parameters based on orientation bands and frequency bands associated with the sets of transform coefficients; and
generate at least a first feature based on probability distribution parameters from a first band of the orientation bands and frequency bands; and

determine an image evaluation indicator associated with the valid image based on a mapping of the generated features to the image evaluation indicator.

14. The system of claim 13 , wherein the central processor to determine the candidate image is the valid image comprises the central processor to determine a probability the candidate image is associated with a predetermined target category based on natural scene statistics associated with the candidate image, perform the category check by comparing the probability to a threshold, apply a flat region detector to the candidate image to determine a percentage of the candidate image including flat regions, and perform the composition check by comparing the percentage to a second threshold.

15. The system of claim 13 , wherein the central processor is further to determine a camera evaluation indicator associated with a camera used to capture the valid image based on a plurality of image evaluation indicators comprising the image evaluation indicator.

16. The system of claim 13 , wherein the sets of transform coefficients comprise discrete cosine transform (DCT) coefficients, the probability distribution comprises a generalized Gaussian distribution (GDD), and the first feature comprises a mean of the probability distribution parameters from the first band.

17. The system of claim 13 , wherein the image quality based feature comprises at least one of a sharpness, a noise, a dynamic range, or an illumination of the valid image.

18. The system of claim 13 , further comprising:
a camera to capture the candidate image; and
a display to present one or more image capture indicators for capturing the candidate image, wherein the image capture indicators comprise at least one of a scene composition indicator, an illumination indicator, or an example image.

19. The system of claim 13 , wherein the system comprises at least one of a personal computer system or a cloud computing system.

20. At least one non-transitory machine readable medium comprising a plurality of instructions that, in response to being executed on a device,

Assignees

• Intel Corp
• Santa Clara

Inventors

• Saad Michele A
• Jaladi Ramesh V
• Corriveau Philip J
• Da Silva Ralston A

Classifications

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• G06T7/0002 Primary

  Inspection of images, e.g. flaw detection · CPC title

• H04N23/631

  Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters · CPC title

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title

• H04N23/633

  for displaying additional information relating to control or operation of the camera · CPC title