Method for dynamic creation of collages from mobile video

Patent metadata

Field	Value
Publication number	US-11880918-B2
Application number	US-201717598079-A
Country	US
Kind code	B2
Filing date	Jul 27, 2017
Priority date	Jul 28, 2016
Publication date	Jan 23, 2024
Grant date	Jan 23, 2024

Abstract

A method and system for automated creation of collages from a video sequence is provided. The system and method dynamically extracts key frames from a video sequence, which are used to create the collages shown on a display. By changing the position of cursors, a new set of key frames is extracted and the content of the collage will change correspondingly reflecting the changes in the key frames selection. The method also includes user interface elements to allow a user to change the layout design (e.g., rotate, overlap, white space, image scaling) of the video frames on the collage. In addition, the method includes different ways for browsing the video sequence by using automatically detected semantic concepts in various frames as indexing points. Further, the method includes defining swiping attributes based on motion characteristics (e.g., zoom/pan, motion of object), audio activity, or facial expression for browsing the video sequence.

First claim

What is claimed is:

1. A non-transitory computer-readable medium with computer-executable instructions stored thereon executed by one or more processors to perform a method for dynamically creating collages from video sequences within a window displayed on a user device, the method comprising:
displaying a collection of captured or downloaded video sequences;
selecting a video sequence from the set of captured or downloaded video sequences, the selected video sequence including a plurality of video frames having a start frame and an end frame;
displaying a visualization of the plurality of video frames, with a first side of the display corresponding to the start frame, and a second side of the display corresponding to the end frame of the selected video sequence;
adjusting the start and end frames of the selected video sequence in response to movement of a cursor along a length of the video sequence;
extracting a plurality of key frames from the selected video segment;
generating a collage layout aspect using the extracted key frames;
displaying the collage on the user device; and optionally
printing or sharing the collage on a social media network.

2. The computer-readable medium of claim 1 , further comprising extracting the plurality of key frames from the selected video segment based on indexing criteria.

3. The computer-readable medium of claim 2 , wherein the indexing criteria include at least one of facial features, natural scenes, animals, sky elements, or cityscapes.

4. The computer-readable medium of claim 2 , wherein the indexing criteria comprise frames containing a face of a particular person.

5. The computer-readable medium of claim 2 , wherein the indexing criteria comprise frames containing a similar color histogram.

6. The computer-readable medium of claim 2 , wherein the indexing criteria further comprise frames containing a particular object.

7. The computer-readable medium of claim 2 , wherein a best key frame is displayed to the user from the plurality of frames selected through the indexing criteria.

8. The computer-readable medium of claim 1 , further comprising extracting the plurality of key frames from the selected video segment based on swiping attributes.

9. The computer-readable medium of claim 8 wherein key frames are located based on motion characteristics of the video sequence.

10. The computer-readable medium of claim 9 , wherein the motion characteristics include locating key frames containing zoom or fast pan action.

11. The computer-readable medium of claim 1 , wherein the swiping attributes include at least one of a speed of a moving video object, audio activities, appearances of faces, or expression of a person.

12. The computer-readable medium of claim 11 , wherein the plurality of key frames is selected based on swiping attributes specifying a speed threshold, volume threshold, or pre-defined expression choice.

13. The computer-readable medium of claim 1 , wherein the layout aspect of the video frames on the collage is modified in response to user selected options.

14. The computer-readable medium of claim 1 , wherein one or more of rotation, overlap, white space, and image scaling features are modified in the video frames.

15. A method for dynamically creating collages from video sequences within a window displayed on a user device, the method comprising:
displaying a collection of captured or downloaded video sequences on a display window of a user device;
selecting a video sequence from the collection of captured or downloaded video sequences, the selected video sequence including a plurality of video frames having a start frame and an end frame;
displaying a visualization of the plurality of video frames, with a first side of the display corresponding to the start frame, and a second side of the display corresponding to the end frame of the selected video sequence;
adjusting the start and end frames of the selected video sequence in response to movement of a cursor along a length of the video sequence;
extracting a plurality of key frames from the selected video segment;
generating a collage layout aspect using the extracted key frames;
displaying the collage on the user device; and optionally
printing or sharing the collage on a social media network.

16. The method of claim 15 , further comprising extracting the plurality of key frames from the selected video segment based on indexing criteria.

17. The method of claim 16 , wherein the indexing criteria include at least one of facial features, natural scenes, animals, sky elements, or cityscapes.

18. The method of claim 16 wherein the indexing criteria comprise frames containing a face of a particular person.

19. The method of claim 16 , wherein the indexing criteria comprise frames containing similar color histogram.

20. The method of claim 16 , wherein the indexing criteria further comprise frames containing a particular object.

21. The method of claim 16 , wherein the best key frame is displayed to the user from the plurality of key frames selected through the indexing criteria.

22. The method of claim 15 , further comprising extracting a plurality of key frames from the selected video segment based on swiping attributes.

23. The method of claim 22 wherein the swiping attributes include at least one of the speed of a moving video object, audio activities, appearances of faces and expression of a person.

24. The method of claim 23 , wherein the plurality of key frames is selected based on swiping attributes specifying a speed threshold, volume threshold, or pre-defined expression choice.

25. The method of claim 23 wherein key frames are located based on motion characteristics of the video sequence.

26. The method of claim 25 , wherein the motion characteristics of the video include locating key frames containing zoom or fact pan action.

27. The method of claim 15 , wherein the layout aspect of the video frames on the collage is modified in response to selected options.

28. The method of claim 15 , wherein one or more of overlap, white space, and image scaling features are modified in the video frames.

Assignees

• Kodak Alaris Inc

Inventors

• Loui Alexander C
• Whitby Laura R
• Manico Joseph A

Classifications

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• G06F3/04845

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06V10/40

  Extraction of image or video features · CPC title

• G06V20/41

  Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49) · CPC title

• G06F16/74 Primary

  Browsing; Visualisation therefor (end-user interfaces for requesting or interacting with video content, e.g. video on demand interfaces or electronic programme guides, H04N21/472) · CPC title