Automatic content-aware collage

Patent metadata

Field	Value
Publication number	US-11615507-B2
Application number	US-202016862424-A
Country	US
Kind code	B2
Filing date	Apr 29, 2020
Priority date	Apr 29, 2020
Publication date	Mar 28, 2023
Grant date	Mar 28, 2023

Abstract

Techniques and systems are described for automatic content-aware collages. Collage templates are generated based on generated set of initial points. Salient regions are determined within digital images, and the salient regions are matched with cells of a collage template. Chrominance of digital images may be mediated to provide a cohesive color scheme among the digital images, and geometric parameters of digital images may be generated to optimize visible salient regions within cells of the template. A collage is generated incorporating the digital images in corresponding cells of the template.

First claim

What is claimed is:

1. A method comprising:
generating, by a processing device and responsive to receiving a user input, a template including a plurality of cells based on a set of initial points;
determining, by the processing device and for each respective one of a plurality of digital images, a salient region based on a visual saliency of respective pixels in the respective digital image;
generating, by the processing device, a saliency map indicating a center of the salient region for each respective one of the plurality of digital images;
assigning, by the processing device, a weight to the center of the salient region for each respective one of the plurality of digital images;
matching, by the processing device, respective salient regions with respective cells based on a comparison of a shape of the salient region with a shape of the cell using deep learning to maximize salient content in visible regions of matched results by centering the center of the salient region in the cell based on the weight;
generating, by the processing device, a collage as including the plurality of digital images based on the template and the matching; and
outputting, by the processing device, the collage.

2. The method of claim 1 , wherein the set of initial points is randomly generated.

3. The method of claim 1 , wherein the user input includes at least one of a number of regions and a norm distance.

4. The method of claim 3 , wherein the norm distance defines a function space usable to define lengths of vectors.

5. The method of claim 1 , wherein the generating the template includes generating a plurality of templates that each include a respective plurality of cells.

6. The method of claim 5 , wherein each of the plurality of templates is generated using a different respective norm distance.

7. The method of claim 5 , wherein each of the plurality of templates is generated using a different set of initial points.

8. The method of claim 1 , wherein the template includes a Voronoi diagram.

9. The method of claim 1 , wherein each respective salient region is a convex hull.

10. The method of claim 1 , further comprising determining translation and scale parameters for a respective one of the digital images based on a salient region of the respective one of the digital images, and wherein the generating the collage is further based on the translation and scale parameters as applied to the respective one of the digital images.

11. The method of claim 1 , further comprising:
mediating a foreground chrominance, for each respective one of the plurality of digital images, based on foreground regions of the respective digital image and foreground regions of a target digital image; and
mediating a background chrominance, for each respective one of the plurality of digital images, based on background regions of the respective digital image and background regions of the target digital image.

12. The method of claim 1 , further comprising:
generating a plurality of collages as each including the plurality of digital images based on a respective template and respective matchings;
displaying the plurality of collages on a display device of the processing device;
receiving another user input to select one of the plurality of collages; and
wherein the outputting the collage includes outputting the selected one of the plurality of collages.

13. The method of claim 1 , further comprising detecting a face in the salient region and assigning a greater weight to the face than the center of the salient region.

14. A system comprising:
a template generation system configured to generate a template including a plurality of cells based on a set of initial points and responsive to receiving a user input to initiate creation of a collage, each cell of the plurality of cells being a complex shape;
an image saliency system configured to determine, for each respective one of a plurality of digital images, a salient region based on a visual saliency of respective pixels in the respective digital image and to generate a saliency map indicating a center of the salient region for each respective one of the plurality of digital images;
a weighting system configured to assign a weight to the center of the salient region for each respective one of the plurality of digital images;
a shape matching system configured to match respective salient regions having complex shapes as non-regular polygons with respective cells having complex shapes as non-regular polygons based on a comparison of the salient region with the cell to maximize salient content in visible regions of matched results by centering the center of the salient region in the cell based on the weight;
a media content editing application configured to generate the collage as including the plurality of digital images based on the template and the matched regions.

15. The system of claim 14 , wherein the user input includes at least one of a number of regions and a norm distance that defines a function space usable to define lengths of vectors.

16. The system of claim 14 , wherein the generating the template includes generating a plurality of templates that each include a respective plurality of cells.

17. The system of claim 16 , wherein each of the plurality of templates is generated using a different respective norm distance.

18. A system comprising:
means for generating, responsive to receiving a user input to initiate creation of a collage, a template including a plurality of cells based on a set of initial points;
means for determining, for each respective one of a plurality of digital images, a salient region based on a visual saliency of respective pixels in the respective digital image;
means for generating a saliency map indicating a center of the salient region for each respective one of the plurality of digital images;
means for assigning a weight to the center of the salient region for each respective one of the plurality of digital images;
means for matching respective salient regions with respective cells based on a comparison of moment features as a quantitative measure of a shape of a function of pixel intensities of the salient region with moment features as a quantitative measure of a shape of a function of pixel intensities of the cell to maximize salient content in visible regions of matched results by centering the center of the salient region in the cell based on the weight;
means for generating the collage as including the plurality of digital images based on the template and the matching.

19. The system of claim 18 , wherein the user input includes a norm distance that defines a function space usable to define lengths of vectors.

20. The system of claim 18 , wherein the means for generating the template includes means for generating a plurality of templates that each include a respective plurality of cells using a different respective norm distance.

Assignees

• Adobe Inc

Inventors

• Mironic{Hacek Over (A)} Ionuţ
• Bîrhal{Hacek Over (A)} Andreea

Classifications

• G06F3/04845

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06T7/0002

  Inspection of images, e.g. flaw detection · CPC title

• G06T3/4038 Primary

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T7/97

  Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85) · CPC title

• G06T2207/20221

  Image fusion; Image merging · CPC title