Automatic creation of media collages

Patent metadata

Field	Value
Publication number	US-10692259-B2
Application number	US-201615380456-A
Country	US
Kind code	B2
Filing date	Dec 15, 2016
Priority date	Dec 15, 2016
Publication date	Jun 23, 2020
Grant date	Jun 23, 2020

Abstract

Techniques for automatic creation of media collages are described. In one or more implementations, unwanted frames are identified and removed from items of media content. A media score is then determined for items of media content based on characteristics of an appearance of the items within a plurality of collage templates. A template score is determined for each collage template of the plurality of collage templates by combining the media scores for each media item of the plurality of media items included in a collage template. At least one of the plurality of collage templates is selected based on determined template scores. Then, at least one media collage is outputted based on the selected collage templates.

First claim

The invention claimed is:

1. In a digital media environment for generating a media collage, a method implemented by at least one computing device, the method comprising:
determining, by the at least one computing device, media scores for each item of a plurality of items of media content, the media scores based on numeric relevance factors that represent characteristics of an appearance of each said item within a respective collage template of a plurality of collage templates, the items of media content including videos, and the characteristics of the appearance including at least a duration of each of the videos, wherein assignment of a respective relevance factor to a respective video is based on how similar the duration of the video is to a respective duration of each of the other videos of the plurality of items of media content;
determining, by the at least one computing device, a template score for each collage template of the plurality of collage templates by combining the media scores for each item of the plurality of items included in the respective said collage template;
selecting, by the at least one computing device, at least one of the plurality of collage templates based on the determined template scores; and
outputting, by the at least computing device, the media collage based on the selected at least one of the plurality of collage templates.

2. The method of claim 1 , wherein the characteristics of the appearance further include an aspect shared between the items of media content.

3. The method of claim 1 , wherein the determining the media scores for the plurality of items of media content further comprises computing a weighted mean of two or more of the relevance factors.

4. The method of claim 1 , wherein the characteristics of the appearance further include a number of objects or faces detected in a pause frame of the item of media content.

5. The method of claim 1 , wherein the assignment of the respective relevance factor is further based on a number of objects or faces detected in a pause frame of the respective video.

6. The method of claim 1 , wherein the assignment of the respective relevance factor is further based on a number of interesting frames in the respective item of media content.

7. The method of claim 1 , wherein the plurality of items of media content are a subset of a larger collection of items of media content, and wherein the plurality of collage templates are a subset of a larger collection of collage templates.

8. The method of claim 1 , further comprising removing, automatically by the at least one computing device, unwanted frames from the videos of the items of media content before the determining of the media scores.

9. The method of claim 1 , wherein the plurality of items of media content further include images.

10. The method of claim 1 , wherein the characteristics of the appearance further include a determined level of quality of pause frames of the items of media content.

11. In a digital media environment for generating media collages comprising multiple media content items, a computing device comprising:
means for determining media scores for a plurality of items of media content based on numeric relevance factors that represent characteristics of an appearance of each of the items of media content in a plurality of collage templates, the items of media content including videos, and the characteristics of the appearance including at least a duration of each of the videos, wherein assignment of a respective relevance factor to a respective video is based on how similar the duration of the video is to a respective duration of each of the other videos of the plurality of items of media content;
means for determining template scores for the plurality of collage templates by combining the media scores of particular items of media content of the plurality of items of media content as the particular items of media content appear in the plurality of collage templates;
means for selecting one or more of the plurality of collage templates including one or more combinations of items of media content based on the template scores; and
means for outputting the media collage based on the selected at least one of the plurality of collage templates and the plurality of items of media content.

12. The computing device of claim 11 , wherein the characteristics of the appearance further include an aspect shared between the items of media content.

13. The computing device of claim 12 , wherein the numeric relevance factors represent the characteristics in an algorithm that weights the numeric relevance factors in accordance with a relative importance of the characteristics represented by the numeric relevance factors.

14. One or more computer-readable storage media comprising instructions stored thereon that, responsive to execution by a computing device, causes the computing device to perform operations comprising:
determining media scores for each item of a plurality of items of media content, the media scores based on numeric relevance factors that represent characteristics of an appearance of each said item within a respective collage template of a plurality of collage templates, the items of media content including videos, and the characteristics of the appearance including at least a duration of each of the videos, wherein assignment of a respective relevance factor to a respective video is based on how similar the duration of the video is to a respective duration of each of the other videos of the plurality of items of media content;
determining a template score for each collage template of the plurality of collage templates by combining the media scores for each item of the plurality of items included in the respective said collage template;
selecting at least one of the plurality of collage templates based on the determined template scores; and
outputting the media collage based on the selected at least one of the plurality of collage templates.

15. The one or more computer-readable storage media of claim 14 , wherein the characteristics of the appearance further include:
a number of objects or faces detected in pause frames of the items of media content;
a determined level of quality of the pause frames of the items of media content;
a number of interesting frames in the items of media content, the interesting frames comprising at least one face or object;
a duration of a subset of the items of media content that are to be displayed sequentially in a single cell of a particular collage template of the plurality of collage templates, the subset of the items of media content determined based on metadata shared between the items of media content in the subset; and
an aspect shared between the items of media content.

16. The one or more computer-readable storage media of claim 15 , wherein the determining the media scores for the plurality of items of media content further comprises computing a weighted mean of two or more of the relevance factors.

17. The one or more computer-readable storage media of claim 15 , wherein the determining the media scores for the plurality of items of media content further comprises applying weights to the relevance factors based on a relative importance of each of the characteristics, one to another.

18. The one or more computer-readable storage media of claim 14 , wherein the characteristics of the appearance further include a number of objects or faces detected in a pause frame of the item of media content.

19. The one or more computer-readable storage me

Assignees

• Adobe Inc

Inventors

• Shah Abhishek
• Bhatt Sameer

Classifications

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title