Methods and systems for differentiating synthetic and non-synthetic images

Patent metadata

Field	Value
Publication number	US-9280723-B2
Application number	US-201414526190-A
Country	US
Kind code	B2
Filing date	Oct 28, 2014
Priority date	Feb 28, 2013
Publication date	Mar 8, 2016
Grant date	Mar 8, 2016

Abstract

The techniques introduced here include a system and method for transcoding multimedia content based on the results of content analysis. The determination of specific transcoding parameters, used for transcoding multimedia content, can be performed by utilizing the results of content analysis of the multimedia content. One of the results of the content analysis is the determination of image type of any images included in the multimedia content. The content analysis uses one or more of several techniques, including analyzing content metadata, examining colors of contiguous pixels in the content, using histogram analysis, using compression distortion analysis, analyzing image edges, or examining user provided inputs. Transcoding the multimedia content can include adapting the content to the constraints in delivery and display, processing and storage of user computing devices.

First claim

What is claimed is:

1. A method for comprising:
receiving, by an analysis module executed by a processor of a computer system, an image to be classified;
analyzing the received image to produce analyzing results, the analyzing using one or more of:
compression distortion analysis, wherein the compression distortion analysis includes analysis of distortion between the received image and a processed version of the received image, the processed version of the received image resulting from a compression and subsequent decompression of the received image;
contiguous pixels analysis, wherein the contiguous pixels analysis includes computing whether a number of pixel groups from the received image, each pixel group defined by a similarity of color in at least a selected number of contiguous pixels, is above a color continuity threshold, wherein the color continuity threshold corresponds to a distinction in image types;
edge detect analysis, wherein the edge detect analysis includes computing whether an edge analysis of the received image resulted in an edge threshold indicating edge geometry or distribution, wherein the edge threshold corresponds to a distinction in image types; or
color histogram analysis, wherein the color histogram analysis includes identifying whether a periodicity threshold is reached in a computation of a number of pixels of the received image, discretized by color, that fall within a list of color ranges that span a color space of the received image, wherein the periodicity threshold corresponds to a distinction in image types; and

classifying the received image with an image type, wherein the image type is at least one of: a synthetic image type or a natural image type, wherein a synthetic image type is for images generated in part by capturing reflected light and a natural image type is for images that are computer generated.

2. The method of claim 1 , wherein analyzing the received image comprises the contiguous pixels analysis.

3. The method of claim 2 , wherein the contiguous pixels analysis comprises identifying that a first pixel of the contiguous pixels and a second pixel of the contiguous pixels are of similar color when one or more of a hue value or a saturation value associated with the first pixel and the second pixel are within a specified threshold level of each other.

4. The method of claim 1 , wherein analyzing the received image comprises the color histogram analysis.

5. The method of claim 1 , wherein analyzing the received image comprises the compression distortion analysis.

6. The method of claim 5 , wherein the analysis of distortion between the received image and the processed version of the received image is performed using one or more of:
a Peak Signal-to-Noise Ratio (PSNR) index; or
a structural similarity (SSIM) index.

7. The method of claim 5 , further comprising:
identifying that the image type is the synthetic image type if the distortion is greater than a specified threshold; and
identifying that the image type is the natural image type if the distortion is less than the specified threshold.

8. The method of claim 1 , wherein analyzing the received image comprises the edge detect analysis.

9. The method of claim 1 , further comprising identifying a plurality of parameters for transcoding the received image based on the image type classification of the received image.

10. The method of claim 1 further comprising identifying a transcoding process to apply to the received image by:
identifying whether a compression is to be applied to the received image based on the image type classification of the received image;
identifying a type of compression to be applied to the received image based on the image type classification of the received image;
identifying one or more of a type or intensity of size reduction to be applied to the received image based on the image type classification of the received image; or
identifying one or more of a type or intensity of color reduction to be applied to the received image based on the image type classification of the received image.

11. The method of claim 1 , wherein:
the image type is the natural image type; and
the received image is one or more of:
a photograph;
a digital version of the photograph, wherein the digital version is generated by scanning the photograph; or
a camera-produced image.

12. A system comprising:
a memory;
one or more processors;
an interface configured to receive an image to be classified;
an analysis module, executed by the one or more processors, configured to analyze the received image to produce analyzing results, the analyzing using one or more of:
compression distortion analysis, wherein the compression distortion analysis includes analysis of distortion between the received image and a processed version of the received image, the processed version of the received image resulting from a compression and subsequent decompression of the received image;
contiguous pixels analysis, wherein the contiguous pixels analysis includes computing whether a number of pixel groups from the received image, each pixel group defined by a similarity of color in at least a selected number of contiguous pixels, is above a color continuity threshold, wherein the color continuity threshold corresponds to a distinction in image types;
edge detect analysis, wherein the edge detect analysis includes computing whether an edge analysis of the received image resulted in an edge threshold indicating edge geometry or distribution, wherein the edge threshold corresponds to a distinction in image types; or
color histogram analysis, wherein the color histogram analysis includes identifying whether a periodicity threshold is reached in a computation of a number of pixels of the received image, discretized by color, that fall within a list of color ranges that span a color space of the received image, wherein the periodicity threshold corresponds to a distinction in image types; and

a classification module, executed by the one or more processors, configured to classify the received image with an image type, wherein the image type is at least one of: a synthetic image type or a natural image type, wherein a synthetic image type is for images generated in part by capturing reflected light and a natural image type is for images that are computer generated.

13. The system of claim 12 , wherein the analysis module uses the contiguous pixels analysis.

14. The system of claim 13 , wherein the contiguous pixels analysis comprises identifying that a first pixel of the contiguous pixels and a second pixel of the contiguous pixels are of similar color when one or more of a hue value or a saturation value associated with the first pixel and the second pixel are within a specified threshold level of each other.

15. The system of claim 12 , wherein the analysis module uses the color histogram analysis or the edge detect analysis.

16. The system of claim 12 , wherein the analysis module uses the compression distortion analysis.

17. The system of claim 16 , wherein the analysis of distortion between the received image and the processed version of the received image is performed using one or more of:
a Peak Signal-to-Noise Ratio (PSNR) index; or
a structural similarity (SSIM) index.

18. The system of claim 16 , wherein the classification module is further configured to:
identify that the image type is the synthetic image type if the distortion is greater than a specified threshold; and
identify that the image type is the natural image type if the distor

Assignees

• Facebook Inc

Inventors

• Lerios Apostolos
• Stoop Dirk
• Mack Ryan
• Bourdev Lubomir Dimitrov
• Paluri Balmanohar

Classifications

• G06T9/00 Primary

  Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) · CPC title

• G06F18/24

  Classification techniques · CPC title

• H04N19/85

  using pre-processing or post-processing specially adapted for video compression · CPC title

• G06K9/6267 Primary

  Physics · mapped topic

• H04N19/40

  using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream · CPC title