Multimedia integration description scheme, method and system for MPEG-7

Patent metadata

Field	Value
Publication number	US-9239877-B2
Application number	US-201113169330-A
Country	US
Kind code	B2
Filing date	Jun 27, 2011
Priority date	Feb 1, 1999
Publication date	Jan 19, 2016
Grant date	Jan 19, 2016

Abstract

The invention provides a system and method for integrating multimedia descriptions in a way that allows humans, software components or devices to easily identify, represent, manage, retrieve, and categorize the multimedia content. In this manner, a user who may be interested in locating a specific piece of multimedia content from a database, Internet, or broadcast media, for example, may search for and find the multimedia content. In this regard, the invention provides a system and method that receives multimedia content and separates the multimedia content into separate components which are assigned to multimedia categories, such as image, video, audio, synthetic and text. Within each of the multimedia categories, the multimedia content is classified and descriptions of the multimedia content are generated. The descriptions are then formatted, integrated, using a multimedia integration description scheme, and the multimedia integration description is generated for the multimedia content. The multimedia description is then stored into a database. As a result, a user may query a search engine which then retrieves the multimedia content from the database whose integration description matches the query criteria specified by the user. The search engine can then provide the user a useful search result based on the multimedia integration description.

First claim

What is claimed is:

1. A method comprising:
generating multimedia object descriptions, from multimedia objects in multimedia content, for an identified multimedia type of multimedia content;
generating non-hierarchical entity relation graph descriptions based on the multimedia object descriptions, wherein the non-hierarchical entity relation graph descriptions relate to communication between the multimedia objects in the multimedia content;
generating an integration description scheme which creates, when implemented, relationships between the multimedia object descriptions and the non-hierarchical entity relations graph descriptions; and
generating a description record to represent content embedded within the multimedia content by integrating the multimedia object descriptions and the non-hierarchical entity relation graph descriptions, wherein the integrating occurs according to the integration description scheme.

2. The method of claim 1 , wherein the multimedia object descriptions further comprise an object hierarchy element and an entity relation graph element.

3. The method of claim 2 , wherein the one object hierarchy element and the entity relation graph element describe one of spatial, temporal, and semantic relationships between pieces of the content embedded within the multimedia content.

4. The method of claim 1 , wherein the multimedia object descriptions describe one of a local and a global multimedia stream.

5. The method of claim 1 , wherein the multimedia object descriptions contain an optional attribute, wherein the optional attribute comprises one of a unique identifier, a reference to another multimedia object description, lists of identifiers of object node elements, and lists of entity node elements that reference the multimedia object element.

6. The method of claim 1 , wherein the multimedia object descriptions further comprise corresponding descriptor elements.

7. The method of claim 6 , wherein the corresponding descriptor elements comprise one of links to external extraction, similarity matching code, and external definitions for descriptions.

8. A system, comprising:
a processor; and
a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
generating multimedia object descriptions, from multimedia objects in multimedia content for an identified multimedia type of multimedia content;
generating non-hierarchical entity relation graph descriptions based on the multimedia object descriptions, wherein the non-hierarchical entity relation graph descriptions relate to communication between the multimedia objects in the multimedia content;
generating an integration description scheme which creates, when implemented, relationships between the multimedia object descriptions and the non-hierarchical entity relations graph descriptions; and
generating a description record to represent content embedded within the multimedia content by integrating the multimedia object descriptions and the non-hierarchical entity relation graph descriptions, wherein the integrating occurs according to the integration description scheme.

9. The system of claim 8 , wherein the multimedia object descriptions further comprise an object hierarchy element and an entity relation graph element.

10. The system of claim 9 , wherein the object hierarchy element and the entity relation graph element describe one of spatial, temporal, and semantic relationships.

11. The system of claim 8 , wherein the multimedia object descriptions describe one of a local and a global multimedia stream.

12. The system of claim 8 , wherein the multimedia object descriptions contain an optional attribute, wherein the optional attribute comprises one of a unique identifier, reference to another multimedia object description, lists of the identifiers of object node elements, and lists of entity node elements that reference the multimedia object element.

13. The system of claim 8 , wherein the multimedia object descriptions further comprise corresponding descriptor elements.

14. The system of claim 13 , wherein the corresponding descriptor elements comprise one of links to external extraction, similarity matching code, and external definitions for descriptions.

15. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:
generating multimedia object descriptions, from multimedia objects in multimedia content for an identified multimedia type of multimedia content;
generating non-hierarchical entity relation graph descriptions based on the multimedia object descriptions, wherein the non-hierarchical entity relation graph descriptions relate to communication between the multimedia objects in the multimedia content;
generating an integration description scheme which creates, when implemented, relationships between the multimedia object descriptions and the non-hierarchical entity relations graph descriptions; and
generating a description record to represent content embedded within the multimedia content by integrating the multimedia object descriptions and the non-hierarchical entity relation graph descriptions, wherein the integrating occurs according to the integration description scheme.

16. The computer-readable storage device of claim 15 , wherein the multimedia object descriptions further comprise an object hierarchy element and an entity relation graph element.

17. The computer-readable storage device of claim 16 , wherein the object hierarchy element and the entity relation graph element describe one of spatial, temporal, and semantic relationships.

18. The computer-readable storage device of claim 15 , wherein the multimedia object descriptions describe one of a local and a global multimedia stream.

19. The computer-readable storage device of claim 15 , wherein the multimedia object contains an optional attribute, wherein the optional attribute comprises one of a unique identifier, reference to another multimedia object description, lists of the identifiers of object node elements, and lists of entity node elements that reference the multimedia object element.

20. The computer-readable storage device of claim 15 , wherein the multimedia object descriptions further comprise corresponding descriptor elements.

Assignees

• Benitez Ana Belen
• Chang Shih-Fu
• Huang Qian
• Paek Seungyup
• Puri Atul
• At & T Ip Ii Lp
• Univ Columbia

Inventors

• Benitez Ana Belen
• Chang Shih-Fu
• Huang Qian
• Paek Seungyup
• Puri Atul

Classifications

• G06F17/3002

  Physics · mapped topic

• G06F17/30038

  Physics · mapped topic

• G06F17/30914

  Physics · mapped topic

• G06F17/30858 Primary

  Physics · mapped topic

• G06F16/48

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title