Multimedia integration description scheme, method and system for MPEG-7

Patent metadata

Field	Value
Publication number	US-9507779-B2
Application number	US-201514967721-A
Country	US
Kind code	B2
Filing date	Dec 14, 2015
Priority date	Feb 1, 1999
Publication date	Nov 29, 2016
Grant date	Nov 29, 2016

Abstract

The invention provides a system and method for integrating multimedia descriptions in a way that allows humans, software components or devices to easily identify, represent, manage, retrieve, and categorize the multimedia content. In this manner, a user who may be interested in locating a specific piece of multimedia content from a database, Internet, or broadcast media, for example, may search for and find the multimedia content. In this regard, the invention provides a system and method that receives multimedia content and separates the multimedia content into separate components which are assigned to multimedia categories, such as image, video, audio, synthetic and text. Within each of the multimedia categories, the multimedia content is classified and descriptions of the multimedia content are generated. The descriptions are then formatted, integrated, using a multimedia integration description scheme, and the multimedia integration description is generated for the multimedia content. The multimedia description is then stored into a database. As a result, a user may query a search engine which then retrieves the multimedia content from the database whose integration description matches the query criteria specified by the user. The search engine can then provide the user a useful search result based on the multimedia integration description.

First claim

We claim:

1. A method comprising:
receiving a multimedia stream having media objects;
categorizing the media objects into a global media object and a local media object, wherein the global media object represents the multimedia stream and is categorized by a first attribute type and the local media object represents a limited feature within the multimedia stream and is categorized by a second attribute type;
receiving the global media object at a global media description generation unit;
receiving the local media object at a local media description generation unit;
generating, at the global media description generation unit, a global multimedia object description for the global media object;
generating, at the local media description generation unit, a local multimedia object description for the local media object;
generating an integration description scheme which creates, when implemented, relationships between the global multimedia object description and the local multimedia object description;
generating a description record to represent a portion of the multimedia stream by integrating, according to the integration description scheme, the global multimedia object description and the local multimedia object description; and
retrieving requested multimedia content based on a search query similarity to the description record.

2. The method of claim 1 , wherein the global multimedia object description and the local multimedia object description further comprise an object hierarchy element and an entity relation graph element.

3. The method of claim 2 , wherein the object hierarchy element and the entity relation graph element describe one of spatial, temporal, and semantic relationships between pieces of content embedded within the multimedia stream.

4. The method of claim 1 , wherein the global multimedia object description and the local multimedia object description contain an optional attribute, wherein the optional attribute comprises one of a unique identifier, a reference to another multimedia object description, lists of identifiers of object node elements, and lists of entity node elements that reference the multimedia objects.

5. The method of claim 1 , wherein the global multimedia object description and the local multimedia object description further comprise corresponding descriptor elements.

6. The method of claim 5 , wherein the corresponding descriptor elements comprise one of links to external extraction, similarity matching code, and external definitions for descriptions.

7. A system comprising:
a processor; and
a computer-readable storage device having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
receiving a multimedia stream having media objects;
categorizing the media objects into a global media object and a local media object, wherein the global media object represents the multimedia stream and is categorized by a first attribute type and the local media object represents a limited feature within the multimedia stream and is categorized by a second attribute type;
receiving the global media object at a global media description generation unit;
receiving the local media object at a local media description generation unit;
generating, at the global media description generation unit, a global multimedia object description for the global media object;
generating, at the local media description generation unit, a local multimedia object description for the local media object;
generating an integration description scheme which creates, when implemented, relationships between the global multimedia object description and the local multimedia object description;
generating a description record to represent a portion of the multimedia stream by integrating, according to the integration description scheme, the global multimedia object description and the local multimedia object description; and

retrieving requested multimedia content based on a search query similarity to the description record.

8. The system of claim 7 , wherein the global multimedia object description and the local multimedia object description further comprise an object hierarchy element and an entity relation graph element.

9. The system of claim 8 , wherein the object hierarchy element and the entity relation graph element describe one of spatial, temporal, and semantic relationships between pieces of content embedded within the multimedia stream.

10. The system of claim 7 , wherein the global multimedia object description and the local multimedia object description contain an optional attribute, wherein the optional attribute comprises one of a unique identifier, a reference to another multimedia object description, lists of identifiers of object node elements, and lists of entity node elements that reference the multimedia objects.

11. The system of claim 7 , wherein the global multimedia object description and the local multimedia object description further comprise corresponding descriptor elements.

12. The system of claim 11 , wherein the corresponding descriptor elements comprise one of links to external extraction, similarity matching code, and external definitions for descriptions.

13. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:
receiving a multimedia stream having media objects;
categorizing the media objects into a global media object and a local media object, wherein the global media object represents the multimedia stream and is categorized by a first attribute type and the local media object represents a limited feature within the multimedia stream and is categorized by a second attribute type;
receiving the global media object at a global media description generation unit;
receiving the local media object at a local media description generation unit;
generating, at the global media description generation unit, a global multimedia object description for the global media object;
generating, at the local media description generation unit, a local multimedia object description for the local media object;
generating an integration description scheme which creates, when implemented, relationships between the global multimedia object description and the local multimedia object description;
generating a description record to represent a portion of the multimedia stream by integrating, according to the integration description scheme, the global multimedia object description and the local multimedia object description; and
retrieving requested multimedia content based on a search query similarity to the description record.

14. The computer-readable storage device of claim 13 , wherein the global multimedia object description and the local multimedia object description further comprise an object hierarchy element and an entity relation graph element.

15. The computer-readable storage device of claim 14 , wherein the object hierarchy element and the entity relation graph element describe one of spatial, temporal, and semantic relationships between pieces of content embedded within the multimedia stream.

16. The computer-readable storage device of claim 13 , wherein the global multimedia object description and the local multimedia object description contain an optional attribute, wherein the optional attribute comprises one of a unique identifier, a reference to another multimedia object description, lists of identifiers of object node elements, and lists of entity node elements that reference the

Assignees

• At & T Ip Ii Lp
• Univ Columbia

Inventors

• Benitez Ana Belen
• Chang Shih-Fu
• Huang Qian
• Paek Seungyup
• Puri Atul

Classifications

• G06F17/30038

  Physics · mapped topic

• Y10S707/99945

  Object-oriented database structure processing · CPC title

• Y10S707/99948

  Application of database or data structure, e.g. distributed, multimedia, or image · CPC title

• G06F17/3002

  Physics · mapped topic

• G06F17/30914

  Physics · mapped topic