Method and apparatus for generating photo-story based on visual context analysis of digital content

Patent metadata

Field	Value
Publication number	US-2016203386-A1
Application number	US-201614994684-A
Country	US
Kind code	A1
Filing date	Jan 13, 2016
Priority date	Jan 13, 2015
Publication date	Jul 14, 2016
Grant date	—

Abstract

A method of generating a photo-story is provided. The method includes generating tags that indicate properties of a context of photo images; predicting, based on the generated tags, scenes indicated by the photo images; and generating, based on the predicted scenes, a photo-story including a combination of the predicted scenes.

First claim

What is claimed is:

1 . A method of generating a photo-story, the method comprising:
generating tags that indicate properties of a context of each of a plurality of photo images; predicting, based on the generated tags, scenes indicated by the photo images; and generating, based on the predicted scenes, at least one photo-story comprising a combination of the predicted scenes.

2 . The method of claim 1 , wherein the generating the tags comprises:
detecting objects in the photo images by using visual pattern recognition models that are learned from training images; and determining, based on the detected objects, the properties of the context of each of the plurality of photo images.

3 . The method of claim 1 , wherein the generating the at least one photo-story comprises:
detecting candidate templates that comprise the predicted scenes, from among a plurality of templates that define a structure of the at least one photo-story; evaluating a matching degree between each of the candidate templates and photo images in the predicted scenes; selecting, based on a result of the evaluating, at least one of the candidate templates as a final template; and generating, based on the selected final template, the at least one photo-story to be provided to a user.

4 . The method of claim 3 , wherein the plurality of templates are preset or automatically generated by using the plurality of photo images and unsupervised machine learning.

5 . The method of claim 1 , further comprising detecting, from among the plurality of photo images, overlapping images that are determined as identical or similar images according to a criterion,
wherein the generating the at least one photo-story comprises selecting, when the detected overlapping images correspond to an identical photo-story, one of the overlapping images as an image to be used in the identical photo-story.

6 . The method of claim 3 , wherein the generating the at least one photo-story comprises downloading at least one image used in a scene of the final template from the Internet.

7 . The method of claim 1 , further comprising collecting the plurality of photo images from at least one of a user terminal, a multimedia content sharing service server, and a social network service server.

8 . An apparatus for generating a photo-story, the apparatus comprising:
a tag generator configured to generate tags that indicate properties of a context of each of a plurality of photo images; a scene predictor configured to predict, based on the generated tags, scenes indicated by the photo images; and a photo-story generator configured to generate, based on the predicted scenes, at least one photo-story comprising a combination of the predicted scenes.

9 . The apparatus of claim 8 , wherein the tag generator comprises:
an object detector configured to detect objects in the photo images by using visual pattern recognition models that are learned from training images; and a properties determiner configured to determine, based on the detected objects, the properties of the context of each of the plurality of photo images.

10 . The apparatus of claim 8 , wherein the photo-story generator comprises:
a template detector configured to detect candidate templates that comprise the predicted scenes, from among a plurality of templates that define a structure of the at least one photo-story; an evaluator configured to evaluate a matching degree between the candidate templates and photo images in the predicted scenes; a selector configured to select, based on a result of the evaluation, at least one of the candidate templates as a final template; and a photo-story creator configured to create, based on the selected final template, the at least one photo-story to be provided to a user.

11 . The apparatus of claim 10 , wherein the plurality of templates are preset or automatically generated by using the plurality of photo images and unsupervised machine learning.

12 . The apparatus of claim 8 , further comprising:
a properties determiner configured to detect, from among the plurality of photo images, overlapping images that are determined as identical or similar images according to a criterion, wherein the photo-story generator selects, when the overlapping images correspond to an identical photo-story, one of the overlapping images as an image to be used in the identical photo-story.

13 . The apparatus of claim 10 , wherein the photo-story creator downloads at least one image used in a scene of the final template from the Internet.

14 . The apparatus of claim 8 , further comprising a network interface configured to receive the plurality of photo images from at least one of a user terminal, a multimedia content sharing service server, and a social network service server.

15 . A method of displaying a photo-story on a user terminal, the method comprising:
displaying a list of photo-stories generated based on tags that indicate properties of a context of a plurality of photo images; and reproducing a photo-story that is selected from the list according to a user input.

16 . The method of claim 15 , wherein the displaying of the list comprises displaying respective titles of the photo-stories together with the tags that indicate properties of photo images included in each of corresponding photo-stories.

17 . The method of claim 15 , wherein the displaying the list comprises displaying the titles hierarchically.

18 . The method of claim 15 , further comprising displaying preferred tags, which are tags that are detected with a frequency higher than a threshold frequency from among the tags of the photo images,
wherein the displaying of the list comprises displaying a list of photo-stories that comprise at least one photo image that is mapped to at least one tag selected from the displayed preferred tags according to a user input.

19 . The method of claim 15 , wherein the reproducing comprises displaying only photo images that are mapped to a tag selected, by a user, from photo images of the selected photo-story.

20 . The method of claim 15 , wherein the reproducing comprises providing, while a photo image that is included in the selected photo-story and another photo-story is displayed, a user interface for jumping to the other photo-story.

21 . A user terminal comprising:
at least one memory device configured to store at least one program for displaying a photo-story; and at least one processor configured to execute the at least one program stored in the at least one memory device, wherein the at least one program comprises instructions for performing a method comprising: displaying a list of photo-stories generated based on tags that indicate properties of a context of a plurality of photo images; and reproducing a photo-story that is selected from the list according to a user input.

22 . The user terminal of claim 21 , wherein the displaying the list comprises displaying respective titles of the photo-stories together with the tags that indicate properties of photo images included in each of corresponding photo-stories.

23 . The user terminal of claim 21 , wherein the displaying the list comprises displaying the titles hierarchically.

24 . The user terminal of claim 21 , wherein the method further comprises displaying preferred tags, which are tags that are detected with a frequency higher than a threshold frequency from among the tags of the ph

Assignees

• Samsung Electronics Co Ltd

Inventors

• Porecki Adam
• Ruta Andrzej
• Labecki Juliusz
• Kudelski Michal
• Rudny Tomasz

Classifications

• H04N5/262

  Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects {; Cameras specially adapted for the electronic generation of special effects} · CPC title

• G06Q50/10 Primary

  Services · CPC title

• H04N21/8133

  specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video programme · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06K9/4671 Primary

  Physics · mapped topic