Method and device for playing multimedia

Patent metadata

Field	Value
Publication number	US-10565754-B2
Application number	US-201514792045-A
Country	US
Kind code	B2
Filing date	Jul 6, 2015
Priority date	Jul 3, 2014
Publication date	Feb 18, 2020
Grant date	Feb 18, 2020

Abstract

Provided are a device and a method for acquiring music information about music played in an electronic device, acquiring a storyline of the music based on the music information, and acquiring and displaying one or more images based on the storyline.

First claim

What is claimed is:

1. A multimedia play device comprising:
a memory storing instructions; and
at least one processor configured to execute the instructions to:
acquire music information about music;
acquire a storyline of the music based on the music information, wherein the storyline is divided into a plurality of paragraphs based on words about time, a character, or space included in the storyline, and a paragraph comprises a plurality of sentences corresponding to a plurality of keywords;
divide the music into a plurality of music segments corresponding to the plurality of paragraphs into which the storyline is divided based on the words about time, a character, or space included in the storyline;
acquire a plurality of images for each keyword of each sentence in the paragraph corresponding to a music segment, and set a number of the images corresponding to each keyword to be proportional to a play time of the music segment corresponding to each keyword; and
control a display to display the images corresponding to each keyword while the music segment corresponding to each keyword is played.

2. The multimedia play device of claim 1 , wherein the music information comprises at least one from among meta-information and audio fingerprint information of the music,
wherein the storyline is acquired based on the at least one from among the meta-information and the audio fingerprint information.

3. The multimedia play device of claim 1 ,
wherein a plurality of candidate images are acquired based on the plurality of sentences, and the plurality of images for each keyword are acquired from the plurality of candidate images.

4. The multimedia play device of claim 1 , wherein the at least one processor is further configured to execute the instructions to acquire emotion information associated with the music based on the storyline,
wherein the plurality of images are acquired based on the emotion information.

5. The multimedia play device of claim 1 , wherein the plurality of sentences form the paragraph based on a similarity.

6. The multimedia play device of claim 1 , wherein the storyline is lyrics of the music,
wherein the music information comprises lyric information about the lyrics, and the plurality of images are displayed based on the lyric information.

7. The multimedia play device of claim 1 , wherein the at least one processor is further configured to execute the instructions to acquire emotion information based on the storyline,
wherein the plurality of images are acquired based on the emotion information.

8. The multimedia play device of claim 6 , wherein the plurality of images are rendered based on emotion information.

9. The multimedia play device of claim 1 , wherein the at least one processor is further configured to execute the instructions to:
acquire a plurality of candidate images based on the plurality of keywords of the plurality of sentences in the paragraph corresponding to the music segment;
determine priorities between the plurality of candidate images based on a preset criterion,
wherein the plurality of images are acquired from the plurality of candidate images based on the determined priorities, and the preset criterion is at least one from among orders between the plurality of candidate images and a number of times that a candidate image is repeatedly provided.

10. The multimedia play device of claim 1 , wherein when the storyline includes a music video, the plurality of images are acquired from one or more images from frames of the music video.

11. The multimedia play device of claim 1 , wherein the at least one processor is further configured to execute the instructions to:
acquire a plurality of candidate images based on the plurality of keywords of the plurality of sentences in the paragraph corresponding to the music segment;
determine priorities between the plurality of candidate images based on a preset criterion,
wherein the plurality of images are acquired from the plurality of candidate images based on the determined priorities.

12. A method of playing a multimedia, the method comprising:
acquiring music information about music;
acquiring a storyline of the music based on the music information, wherein the storyline is divided into a plurality of paragraphs based on words about time, a character, or space included in the storyline, and a paragraph comprises a plurality of sentences corresponding to a plurality of keywords;
dividing the music into a plurality of music segments corresponding to the plurality of paragraphs into which the storyline is divided based on the words about time, a character, or space included in the storyline;
acquiring a plurality of images for each keyword of each sentence in the paragraph corresponding to a music segment, and setting a number of the images corresponding to each keyword to be proportional to a play time of the music segment corresponding to each keyword; and
displaying the images corresponding to each keyword while the music segment corresponding to each keyword is played.

13. The method of claim 12 , wherein the music information comprises at least one from among meta-information and audio fingerprint information of the music,
wherein the storyline is acquired based on the at least one from among the meta-information and the audio fingerprint information.

14. The method of claim 12 ,
wherein a plurality of candidate images are acquired based on the plurality of sentences, and the plurality of images for each keyword are acquired from the plurality of candidate images.

15. The method of claim 12 , further comprising:
acquiring emotion information based on the storyline,
wherein the plurality of images are acquired based on the emotion information.

16. A non-transitory computer-readable recording medium having recorded thereon a program for realizing the method of claim 12 .

17. The method of claim 12 , wherein the method further comprises:
acquiring a plurality of candidate images based on the plurality of keywords of the plurality of sentences in the paragraph corresponding to the music segment; and
determining priorities between the plurality of candidate images based on a preset criterion,
wherein the plurality of images are acquired from the plurality of candidate images based on the determined priorities, and the preset criterion is at least one from among orders between the plurality of candidate images and a number of times that a candidate image is repeatedly provided.

18. The method of claim 12 , wherein when the storyline includes a music video, the plurality of images are acquired from one or more images from frames of the music video.

19. The method of claim 12 , wherein the method further comprises:
acquiring a plurality of candidate images based on the plurality of keywords of the plurality of sentences in the paragraph corresponding to the music segment; and
determining priorities between the plurality of candidate images based on a preset criterion,
wherein the plurality of images are acquired from the plurality of candidate images based on the determined priorities.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Zeng Chengbin
• Shan Yuxiang
• Chen Youxin

Classifications

• G06F16/5866

  using information manually generated, e.g. tags, keywords, comments, manually generated location and time information · CPC title

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• G11B27/11 Primary

  by using information not detectable on the record carrier · CPC title

• H04N21/8113

  comprising music, e.g. song in MP3 format · CPC title

• H04N21/4722

  for requesting additional data associated with the content · CPC title