System and method for multimedia content composition

Patent metadata

Field	Value
Publication number	US-2016180379-A1
Application number	US-201414575211-A
Country	US
Kind code	A1
Filing date	Dec 18, 2014
Priority date	Dec 18, 2014
Publication date	Jun 23, 2016
Grant date	—

Abstract

In one embodiment, a method of blending multimedia with a multimedia composition system may include generating a swap list by identifying potential match points of objects, scenes, or assets, in a multimedia template that are capable of being replaced with personal multimedia assets (PMA) in a user's collection, generating a composition list by designating particular PMA from the PMA in the user's collection to replace at least a portion of the objects, scenes, or assets in the multimedia template identified in the swap list by matching metadata of the particular PMA with metadata of the objects, scenes, or assets in the multimedia template, and composing blended multimedia using the composition list by swapping the objects, scenes, or assets in the multimedia template with the particular PMA matched based on the respective metadata.

First claim

1 . A method of blending multimedia with a multimedia composition system, comprising:
generating a swap list by identifying potential match points of objects, scenes, or assets, in a multimedia template that are capable of being replaced with personal multimedia assets (PMA) in a user's collection; generating a composition list by designating particular PMA from the PMA in the user's collection to replace at least a portion of the objects, scenes, or assets in the multimedia template identified in the swap list by matching metadata of the particular PMA with metadata of the objects, scenes, or assets in the multimedia template; and composing blended multimedia using the composition list by swapping the objects, scenes, or assets in the multimedia template with the particular PMA matched based on the respective metadata.

2 . The method of claim 1 , wherein the multimedia template comprises commercial multimedia content or a virtual environment.

3 . The method of claim 2 , wherein generating the swap list is performed by a front-end processor component using the commercial multimedia content, generating the composition list is performed by a user agent component using the swap list, and composing the blended multimedia is performed by a session composer component using the composition list.

4 . The method of claim 1 , wherein generating the swap list comprises parsing a metadata table received with the multimedia template and separating elements in the metadata table into a table included in the swap list comprising an identified object, scene, or asset in the multimedia template to swap out, a placement of the object, scene, or asset displayed in the multimedia template, and a suggested object, scene, or asset to swap in at the placement from either the PMA or a commercial retailer.

5 . The method of claim 1 , wherein matching metadata of the multimedia template with metadata of the particular PMA comprises comparing attributes of the metadata of the multimedia template to attributes of the metadata of the particular PMA and determining whether a threshold amount of attributes match.

6 . The method of claim 1 , wherein the PMA comprises still images, two-dimensional (2D) images, three-dimensional (3D) models, videos, animations, audio, text, or some combination thereof.

7 . The method of claim 1 , wherein composing the blended multimedia comprises fetching the particular PMA identified in the composition list from one or more data repositories, composing objects in the particular PMA that has been fetched, identifying visual objects included in the multimedia template based on metadata, graphically blending, overlaying, or some combination thereof, the composed objects in the particular PMA that has been fetched at its placement identified in the composition list, separating audio in the multimedia template, identifying and substituting caption or text fields with appropriate text from the particular PMA that has been fetched, rendering the blended multimedia, and setting triggers for user interaction.

8 . The method of claim 1 , comprising displaying a graphical-user interface (GUI) that enables configuration of swappable settings comprising enabling swapping, allowing access to a user's music collection, allowing access to a user's photograph collection, allowing access to a user's web usage statistics, and choosing desired commercial retailers for swapped objects.

9 . The method of claim 1 , comprising outputting the blended multimedia for user consumption, to monitor for user interaction, and to trigger appropriate responses based on certain interactions.

10 . A tangible, non-transitory computer-readable medium storing computer instructions thereon, the computer instructions, when executed by a processor, configured to:
generate a swap list by identifying potential match points of objects, scenes, or assets, in a multimedia template that are capable of being replaced with personal multimedia assets (PMA) in a user's collection; generate a composition list by designating particular PMA to replace at least a portion of the objects, scenes, or assets in the multimedia template identified in the swap list by matching metadata of the particular PMA with metadata of the objects, scenes, or assets in the multimedia template; and compose blended multimedia using the composition list by swapping the objects, scenes, or assets in the multimedia template with the particular PMA matched based on the respective metadata.

11 . The computer-readable medium of claim 10 , wherein the multimedia template comprises a virtual environment, the composition list comprises a scene list including PMA and advertisements based on a user's personal information and preferences, and the instructions are configured to:
output the blended multimedia for user consumption; monitor a user's enjoyment level of the virtual environment by using facial scanning, neural scanning, galvanic skin response, heart rate detection, breathing rate detection, or some combination thereof; and use machine learning to determine correlations between enjoyment or lack of enjoyment with the blended multimedia and a user's mood.

12 . The computer-readable medium of claim 11 , wherein the instructions are configured to generate a different scene list with different PMA and advertisements when the lack of enjoyment is determined.

13 . The computer-readable medium of claim 11 , wherein the instructions configured to monitor the user's enjoyment level comprises instructions for receiving signals indicative of a measured parameter from one or more wearables, sensors, controllers, or some combination thereof.

14 . The computer-readable medium of claim 11 , wherein the instructions configured to use machine learning to determine correlations between enjoyment or lack of enjoyment with the blended multimedia comprises instructions for associating the user's enjoyment level or lack of enjoyment level with the PMA and advertisements in the virtual environment.

15 . The computer-readable medium of claim 14 , wherein the instructions are configured to tailor the virtual environment with PMA according to the user's detected mood based on past correlations of PMA and enjoyment level while in that mood.

16 . The computer-readable medium of claim 11 , wherein the instructions are configured to select the advertisements to be included in the scene list when a display priority flag associated with the advertisement is set to true or not null and the advertisements with the display priority flag set to true or not null override any PMA selected at a particular placement in the virtual environment.

17 . The computer-readable medium of claim 11 , wherein the instructions are configured to select the PMA and the advertisements based on a user's web browsing history, web purchasing history, social media profile, viewing habits, listening habits, and psychological profiling.

18 . An electronic device, comprising:
a means for generating a swap list by identifying potential match points of objects, scenes, or assets, in a multimedia template that are capable of being replaced with personal multimedia assets (PMA) in a user's collection; a means for generating a composition list by determining particular PMA of the PMA in the user's collection to replace at least a portion of the objects, scenes, or assets in the multimedia template identified in the swap list by matching metadata of the particular PMA with metadata of the objects, scenes, or assets in the multimedia template; and a means for composing blended multimedi

Assignees

• Nbcuniversal Media Llc

Inventors

• Salmon Michael Clive
• Ng Sheau Bao

Classifications

• G06Q30/0255 Primary

  based on user history · CPC title

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title