Image composition evaluating apparatus, information processing apparatus and methods thereof

Patent metadata

Field	Value
Publication number	US-10204271-B2
Application number	US-201414448968-A
Country	US
Kind code	B2
Filing date	Jul 31, 2014
Priority date	Aug 2, 2013
Publication date	Feb 12, 2019
Grant date	Feb 12, 2019

Abstract

In the present invention, an attribution is extracted from each region obtained by segmentation of an image, relationships among the regions are described, and a composition of the image is evaluated based on the attributions and the relationships.

First claim

What is claimed is:

1. An apparatus for evaluating image composition comprising:
one or more memories that store a set of instructions; and
one or more processors that execute the set of instructions to:
segment an image into a plurality of regions;
extract at least one feature from each of the plurality of regions;
classify each of the plurality of regions into a preset class based on the extracted at least one feature and a trained model;
extract at least one attribution from each of the plurality of regions;
collect the at least one attribution for each of the plurality of regions;
merge isolated regions in the plurality of regions and update the attributions of the merged regions after collecting the at least one attribution for each of the plurality of regions;
describe relationships among the plurality of regions based on the extracted attributions; and
evaluate a composition of the image to determine whether at least one composition problem is included in the image, based on the extracted attributions, the described relationships and at least one preset criterion,
wherein when updating the attributions of the merged regions, if a plurality of regions merged into one region have more than one class, the class for the one merged region is the one that most of the plurality of regions have.

2. The apparatus according to claim 1 , wherein an over-segmentation method is employed to segment the image into the plurality of regions.

3. The apparatus according to claim 2 , wherein the over-segmentation method is a Felzenszwalb method or an SLIC method.

4. The apparatus according to claim 1 , wherein a Recursive Neural Network method is employed to classify each of the plurality of regions into the preset class.

5. The apparatus according to claim 1 , wherein the one or more processors execute the set of instructions to smooth borders between the regions after merging isolated regions and before updating the attributions of the regions.

6. The apparatus according to claim 1 , wherein the one or more processors execute the set of instructions to:
calculate relative positions of the plurality of regions based on distances, neighbourhood and symmetries; and
calculate coordinate degrees among the plurality of regions based on relative area proportions, color matching and region complexity.

7. The apparatus according to claim 1 , wherein the one or more processors execute the set of instructions to find locations and reasons for the at least one composition problem in the image with respect to the at least one preset criterion.

8. The apparatus according to claim 1 , wherein the one or more processors execute the set of instructions to:
output and inform of the at least one composition problem after evaluating the composition of the image based on the extracted attributions, the described relationships and the at least one preset criterion.

9. The apparatus according to claim 1 , wherein the image is a photographic image.

10. The apparatus according to claim 1 , wherein the extracted attributions and the described relationships depend on the at least one preset criterion.

11. The apparatus according to claim 1 , wherein the extracted attributions comprise at least one of class, position, size, color and texture.

12. The apparatus according to claim 1 , wherein the described relationships comprise at least one of relative position, relative area proportion, color matching and texture matching.

13. The apparatus according to claim 1 , wherein the at least one preset criterion comprises at least one of object on head, area size judgement, major object position judgement, kissing border, close to border, middle separation and color matching.

14. The apparatus according to claim 1 , wherein the at least one composition problem includes at least one of an object on head, area size judgement, major object position judgement, kissing border, close to border, middle separation, and color matching.

15. A method for evaluating image composition comprising:
segmenting an image into a plurality of regions;
extracting at least one feature from each of the plurality of regions;
classifying each of the plurality of regions into a preset class based on the extracted at least one feature and a trained model;
extracting at least one attribution from each of the plurality of regions;
collecting the at least one attribution for each of the plurality of regions;
merging isolated regions in the plurality of regions;
updating the attributions of the merged regions after collecting the at least one attribution for each of the plurality of regions, wherein when updating the attributions of the merged regions, if a plurality of regions merged into one region have more than one class, the class for the one merged region is the one that most of the plurality of regions have;
describing relationships among the plurality of regions based on the extracted attributions; and
evaluating a composition of the image to determine whether at least one composition problem is included in the image, based on the extracted attributions, the described relationships and at least one preset criterion.

16. A non-transitory computer-readable medium storing instructions, which when executed by one or more processors, cause the one or more processors to perform a method comprising:
segmenting an image into a plurality of regions;
extracting at least one feature from each of the plurality of regions;
classifying each of the plurality of regions into a preset class based on the extracted at least one feature and a trained model;
extracting at least one attribution from each of the plurality of regions;
collecting the at least one attribution for each of the plurality of regions;
merging isolated regions in the plurality of regions;
updating the attributions of the merged regions after collecting the at least one attribution for each of the plurality of regions, wherein when updating the attributions of the merged regions, if a plurality of regions merged into one region have more than one class, the class for the one merged region is the one that most of the plurality of regions have;
describing relationships among the plurality of regions based on the extracted attributions; and
evaluating a composition of the image to determine whether at least one composition problem is included in the image, based on the extracted attributions, the described relationships and at least one preset criterion.

Assignees

• Canon Kk

Inventors

• Lv you
• Jiang Yong
• Wu Bo
• Li Xian

Classifications

• G06T7/187

  involving region growing; involving region merging; involving connected component labelling · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06K9/00664 Primary

  Physics · mapped topic

• G06T7/0002 Primary

  Inspection of images, e.g. flaw detection · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title