Smart multimedia processing

Patent metadata

Field	Value
Publication number	US-10691879-B2
Application number	US-201514973412-A
Country	US
Kind code	B2
Filing date	Dec 17, 2015
Priority date	Jan 4, 2015
Publication date	Jun 23, 2020
Grant date	Jun 23, 2020

Abstract

Embodiments of the present disclosure intend to provide a method and system for processing multimedia data. According to one aspect of the present disclosure, there is provided a method for processing multimedia data, comprising: generating an annotation text; and associating the annotation text with a corresponding partition in the multimedia data. By using the processing method and system according to the embodiments of the present disclosure, for example, multimedia data searching efficiency may be effectively enhanced.

First claim

The invention claimed is:

1. A method for processing multimedia data into annotated multimedia partitions using a multimedia processing system including one or more processors and memory, the method comprising:
obtaining a first multimedia partition of a first multimedia data stream;
obtaining a second multimedia partition of a second multimedia data stream, the second multimedia partition being different from the first multimedia partition;
generating a first annotation and a second annotation, the second annotation being different from the first annotation;
associating the first annotation with the first multimedia partition of the first multimedia data stream;
associating the second annotation with the second multimedia partition of the second multimedia data stream;
combining the first multimedia data stream and the second multimedia data stream to form a combined multimedia data stream that includes the first multimedia partition associated with the first annotation and the second multimedia partition associated with the second annotation, the first annotation including first annotation text, and the second annotation including second annotation text; and
performing a full-text search of at least the different first and second annotation texts associated with the different first and second multimedia partitions, respectively, for corresponding multimedia data, using the combined multimedia data stream.

2. The method according to claim 1 , wherein the first and second annotation includes information indicating start time and stop time of a corresponding partition.

3. The method according to claim 1 , wherein the first and second annotation comprises information indicating at least one of keyword, content description, importance level, classification information, main color, role attribute, hot degree graph, key area, security level and geographical location of the multimedia data.

4. The method according to claim 1 , wherein the multimedia data is one of video data, audio data, image data, and text data.

5. The method according to claim 4 , wherein when the multimedia data is video data or image data, an optical character recognition technology is used to recognize characters in a video frame or an image, and the first and second annotation is generated based on the recognized characters.

6. The method according to claim 4 , wherein when the multimedia data is audio data, a voice recognition technology is used to recognize words in an audio, and the first and second annotation is generated based on the recognized words.

7. The method according to claim 1 , wherein the first and second annotation comprises information indicating at least one of a displaying location during play and a displaying format.

8. The method according to claim 1 , further comprising:
having performed the full-text search of at least the first and second annotation texts, displaying a corresponding partition of the multimedia data based on a search result.

9. The method according to claim 8 , wherein performing the full-text search of at least the first annotation text and the second annotation text using the combined multimedia data stream comprises:
performing the full-text search further using at least one of a keyword, a content description, an importance level, classification information, a main color, a role attribute, a hot degree graph, a key area, a security level, and a geographical location of the multimedia data.

10. The method according to claim 8 , wherein performing the full-text search of at least the first annotation text and the second annotation text using the combined multimedia data stream is performed only if the first annotation or the second annotation respectively satisfies a threshold security level.

11. The method according to claim 8 , wherein displaying a corresponding partition of the multimedia data based on the search result is only displaying a corresponding partition satisfying a threshold security level in the first or second annotation.

12. The method of claim 1 wherein the obtaining of the first multimedia partition includes obtaining the first multimedia partition that corresponds to a first continuously shot video partition, and wherein the obtaining of the second multimedia partition includes obtaining the second multimedia partition that corresponds to a second continuously shot video partition.

13. The method of claim 12 wherein the combining of the first multimedia data stream and the second multimedia data stream to form the combined multimedia data stream includes forming the combined multimedia data stream that includes the first continuously shot video partition associated with the first annotation and the second continuously shot video partition associated with the second annotation, the first continuously shot video partition and the second continuously shot video partition being discontinuous in time.

14. The method of claim 1 further comprising:
obtaining a respective multimedia partition among the first multimedia partition and the second multimedia partition; and
obtaining a respective annotation among the first annotation and the second annotation, the respective annotation corresponding to the respective multimedia partition, the respective annotation including at least displayable information and a displaying location within the respective multimedia partition where the displayable information is to be displayed during play of the respective multimedia partition.

15. The method of claim 14 further comprising:
playing the respective multimedia partition, the playing of the respective multimedia partition including displaying the displayable information included in the respective annotation at the displaying location within the respective multimedia partition.

16. The method of claim 1 further comprising:
obtaining a respective multimedia partition among the first multimedia partition and the second multimedia partition, the respective multimedia partition including non-stream image data.

17. A system for processing multimedia data into annotated multimedia partitions, comprising:
a database containing multimedia data communicatively coupled to a processor configured to store computer executable code, the computer executable code configured to perform the following steps:
obtaining a first multimedia partition of a first multimedia data stream;
obtaining a second multimedia partition of a second multimedia data stream, the second multimedia partition being different from the first multimedia partition;
generating a first annotation and a second annotation, the second annotation being different from the first annotation;
associating the first annotation with the first multimedia partition of the first multimedia data stream;
associating the second annotation with the second multimedia partition of the second multimedia data stream;
combining the first multimedia data stream and the second multimedia data stream to form a combined multimedia data stream that includes the first multimedia partition associated with the first annotation and the second multimedia partition associated with the second annotation, the first annotation including first annotation text, and the second annotation including second annotation text; and; and
performing a full-text search of at least the different first and second annotation texts associated with the different first and second multimedia partitions, respectively, for corresponding multimedia data, using the combined multimedia data stream.

18. The system according to c

Assignees

• Emc Corp
• Emc Ip Holding Co Llc

Inventors

• Wu Sandro Jiawei
• Chen Chao
• Lu Patrick Minggang
• Xu Scott Shuo
• Liu Jingjing

Classifications

• G06F16/487

  using geographical or spatial information, e.g. location · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• G06F40/169 Primary

  Annotation, e.g. comment data or footnotes · CPC title

• G06F16/43 Primary

  Querying · CPC title