Methods for processing multimedia flows and corresponding devices

Patent metadata

Field	Value
Publication number	US-9609179-B2
Application number	US-201113824685-A
Country	US
Kind code	B2
Filing date	Sep 22, 2011
Priority date	Sep 22, 2010
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

A method for synchronizing of multimedia flows provided by different sources or transmitted via different transport protocols to a single receiving device or are designed to be rendered by different rendering devices. According to the invention, a reference datum representative of a datum of the first multimedia flow is provided in the second multimedia flow and associated with a datum of this second multimedia flow to synchronise the two flows. This reference datum constitutes a timing reference for the datum of the second multimedia flow with which it is associated. It is used to synchronize the two multimedia flows.

First claim

The invention claimed is:

1. A device for receiving multimedia data flows-comprising:
a first receiver that receives a first multimedia flow;
a second receiver that receives a second multimedia flow comprising video data,
said first multimedia flow comprising at least one first audio datum and/or video datum associated with at least one reference datum comprising a video frame and representative of a second video datum of said-second multimedia flow;
a synchronizer that synchronizes said first and said second multimedia flows by using said reference datum, said synchronizer synchronizing said first audio datum and/or video datum with a second video datum of said second multimedia flow represented by said reference datum associated with said; and
a renderer that renders said synchronized first and second multimedia flows.

2. The device according to claim 1 , wherein the synchronizing comprises comparing the video data of the second multimedia flow with the video frame comprised in said reference datum.

3. The device according to claim 1 , wherein the synchronizer calculates a signal-to-noise ratio between each of the video data of the second multimedia flow and the video frame comprised in said reference datum.

4. The device according to claim 1 , wherein said second video datum is a video frame and said video frame comprised in said reference datum is said second video datum or a video frame derived from said second video datum.

5. The device according to claim 4 , wherein said video frame derived from said second video datum comprises a frame corresponding to a low resolution image of said second video datum.

6. The device according to claim 1 , wherein said first and second multimedia flows come from different multimedia sources and/or are provided by different transport networks.

7. A device for receiving a first multimedia data flow comprising:
a capture interface that captures the rendering of a second multimedia flow;
a receiver that receives said first multimedia flow, said first multimedia flow comprising at least one first audio datum and/or video datum associated with at least one reference datum comprising a video frame and representative of a second video datum of said second multimedia flow;
a synchronizer that synchronizes the first multimedia flow with said second multimedia flow by using said reference datum, said synchronizer synchronizing said at least one first video datum and/or audio datum with a second video datum of said second multimedia flow represented by said reference datum associated with said first datum; and
a renderer that renders the synchronized first multimedia flow.

8. The device according to claim 7 , wherein the synchronizing comprises comparing the video data of the second multimedia flow with the video frame comprised in said reference datum.

9. The device according to claim 7 , wherein said second video datum is a video frame and said video frame and said video frame comprised in said reference datum is said second video datum or a video frame derived from said second video datum.

10. The device according to claim 9 , wherein said video frame derived from said second video datum is a frame corresponding to a low resolution image of said second video datum.

11. The device according to claim 7 , wherein said first and second multimedia flows come from different multimedia sources and/or are provided by different transport networks.

12. A transmitting device for generating a first multimedia data flow, said first multimedia flow comprising at least one first audio datum and/or video datum, said first multimedia flow to be synchronized with a received second multimedia flow comprising video data, the transmitting device comprising:
a receiver that receives said second multimedia,
a renderer that renders said second multimedia flow;
a first generator that generates said first multimedia flow, during said rendering of said second multimedia flow, said first datum being generated at the time of rendering of a second audio or video datum of said second multimedia flow;
a second generator that generates at least one reference datum comprising a video frame and being representative of said second rendered video datum and with which said first datum must be synchronized;
an inserter that inserts said reference datum in said first multimedia flow, said inserting comprising introducing said reference datum into said first multimedia flow and associating said reference datum with said first datum; and
a transmitter that transmits said first generated multimedia flow comprising said inserted reference datum.

13. The device according to claim 12 , wherein said second video datum is a video frame and said video frame comprised in said reference datum is said second video datum or a video frame derived from said second video datum.

14. The device according to claim 13 , wherein said video frame derived from said second video datum comprises a frame corresponding to a low resolution image of said second video datum.

15. A method for generating a first multimedia data flow in a processing device, said first multimedia flow having at least one first audio and/or video datum, said first multimedia flow being generated during a rendering of a second multimedia flow having video data, the method comprising:
generating said at least one first audio datum and/or video datum at the time of rendering of a second video datum of said second multimedia flow;
generating at least one reference datum having a video frame representative of said second rendered video datum, with which said at least one first audio datum and/or video datum must be synchronized; and
inserting said at least one reference datum in said first multimedia flow, said inserting comprising introducing said at least one reference datum into said first multimedia flow, and associating said at least one reference datum with said at least one first audio datum and/or video datum, wherein:
said second video datum of said second multimedia flow is a video frame, and said video frame comprised in said at least one reference datum is said second video datum of said second multimedia flow or a video frame derived from said second video datum of said second multimedia flow; and
said video frame derived from said second video datum of said second multimedia flow comprises a frame corresponding to a low resolution image of said second video datum of said second multimedia flow.

16. A device for rendering a first multimedia flow having at least one first audio datum and/or video datum associated with at least one reference datum, said at least one reference datum having a video frame representative of a second video datum of a second multimedia flow having video data, said device comprising:
a synchronizer that synchronizes said first and said second multimedia flows by using said at least one reference datum, said synchronizer synchronizing said at least one first audio and/or video datum of said first multimedia flow with a second video datum of said second multimedia flow represented by said video frame of said at least one reference datum associated with said at least one first audio datum and/or video datum of said first multimedia flow;
a first renderer that renders said synchronized first multimedia flow, a rendering of said at least one first audio datum and/or video datum of said first multimedia flow being synchronous with a rendering of said second video datum of said second multimedia flow; and
a converter that converts said video frame comprised in said at least one reference datum into a same resolutio

Assignees

• Bichot Guillaume
• Laurent Anthony
• Legallais Yvon
• Thomson Licensing

Inventors

• Bichot Guillaume
• Laurent Anthony
• Legallais Yvon

Classifications

• H04N21/4622

  Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet (web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859) · CPC title

• H04N21/44008

  involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• H04N21/435

  Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream · CPC title

• H04N21/4341

  Demultiplexing of audio and video streams · CPC title

• H04N21/41407

  embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop (constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57) · CPC title