VLC-based video frame synchronization

Patent metadata

Field	Value
Publication number	US-10136034-B2
Application number	US-201515544325-A
Country	US
Kind code	B2
Filing date	Jan 23, 2015
Priority date	Jan 23, 2015
Publication date	Nov 20, 2018
Grant date	Nov 20, 2018

Abstract

A VLC signal representing an alignment identifier is detected by cameras ( 4 ) of multiple user devices ( 1, 2, 3 ) filming a scene. Encoded video frames ( 91, 92, 93 ) from the user devices ( 1, 2, 3 ) are decoded and light patterns representing the captured VLC signal are identified in at least some of the video frames following decoding. The light patterns are decoded into alignment identifiers that are used in order to time align video frames ( 91, 92, 93 ) from different user devices ( 1, 2, 3 ) to thereby achieve video frame synchronization of video data from multiple user devices ( 1, 2, 3 ) recording a scene. The embodiments thereby enable video frame synchronization without the need for accurate clock synchronization between the user devices ( 1, 2, 3 ) and a video synchronization system ( 10 ).

First claim

The invention claimed is:

1. A video synchronization method comprising:
modulating an alignment identifier with a modulation signal to get a light control signal;
outputting said light control signal to a light controller connected to a visible light communication, VLC, capable light source and configured to control said VLC-capable light source according to said light control signal to output a VLC signal;
receiving, from each user device of multiple user devices, a bitstream of encoded video frames representing video data captured by a camera of or connected to said user device;
decoding, for each user device of said multiple user devices, encoded video frames of said bitstream to get a stream of decoded video frames;
identifying, for each user device of said multiple user devices and in at least one decoded video frame of said stream of decoded video frames, a light pattern representing a VLC signal captured by said camera of or connected to said user device;
decoding, for each user device of said multiple user device, said light pattern into an alignment identifier; and
time aligning video frames from different user devices among said multiple user devices based on said alignment identifiers.

2. The video synchronization method according to claim 1 , wherein time aligning said video frames comprises time aligning said video frames from said different user devices based on respective values of said alignment identifiers.

3. The video synchronization method according to claim 2 , wherein time aligning said video frames comprises assigning the same output or playback identifier value to video frames from said different user devices having the same or corresponding value of said alignment identifier.

4. The video synchronization method according to claim 2 , wherein time aligning said video frames comprises outputting video frames from said different user devices having the same or corresponding value of said alignment identifier at the same point in time.

5. A video synchronization system, wherein said video synchronization system is configured to:
modulate an alignment identifier with a modulation signal to get a light control signal;
output said light control signal to a light controller connected to a visible light communication, VLC, capable light source and configured to control said VLC-capable light source according to said light control signal to output a VLC signal;
receive, from each user device of multiple user devices, a bitstream of encoded video frames representing video data captured by a camera of or connected to said user device;
decode, for each user device of said multiple user devices, encoded video frames of said bitstream to get a stream of decoded video frames;
identify, for each user device of said multiple user devices and in at least one decoded video frame of said stream of decoded video frames, a light pattern representing a VLC signal captured by said camera of or connected to said user device;
decode, for each user device of said multiple user device, said light pattern into an alignment identifier; and
time align video frames from different user devices among said multiple user devices based on said alignment identifiers.

6. The video synchronization system according to claim 5 , wherein said video synchronization system is configured to time align said video frames from said different user devices based on respective values of said alignment identifiers.

7. The video synchronization system according to claim 6 , wherein said video synchronization system is configured to assign the same output or playback identifier value to video frames from said different user devices having the same or corresponding value of said alignment identifier.

8. The video synchronization system according to claim 6 , wherein said video synchronization system is configured to output video frames from said different user devices having the same or corresponding value of said alignment identifier at the same point in time.

9. The video synchronization system according to claim 5 , wherein
said video synchronization system is configured to identify at least one decoded video frame comprising said light pattern;
said video synchronization system is configured to remove said identified at least one decoded video frame from said stream of decoded video frames; and
said video synchronization system is configured to encode decoded video frames of said stream of decoded video frames into encoded video frames.

10. The video synchronization system according to claim 5 , wherein
said video synchronization system is configured to identify at least one decoded video frame comprising said light pattern;
said video synchronization system is configured to replace said identified at least one decoded video frame with a neighboring decoded video frame lacking said light pattern of said stream of decoded video frames; and
said video synchronization system is configured to encode decoded video frames of said stream of decoded video frames into encoded video frames.

11. The video synchronization system according to claim 5 , wherein
said video synchronization system is configured to measure, for each user device of said multiple user devices, a noise floor of transmission of said bitstream;
said video synchronization system is configured to detect, for each user device of said multiple user devices and based on said measurements, a frequency of said light pattern in the image frequency domain; and
said video synchronization system is configured to decode, for each user device of said multiple user devices and based on information of said frequency of said light pattern, said light pattern into said alignment identifier.

12. The video synchronization system according to claim 5 , wherein
said video synchronization system is configured to obtain a quality of service, QoS, parameter representative of a quality of transmission of said bitstream for at least one user device of said multiple user devices; and
said video synchronization system is configured to control modulation of said alignment identifier and/or generation of said alignment identifier based on a value of said QoS parameter.

13. The video synchronization system according to claim 12 , wherein said video synchronization system is configured to increase time intervals between occasions of modulating said alignment identifier if a value of said QoS parameter represents a worse QoS than a defined minimum QoS or if a value of said quality score represents a worse quality than a defined minimum quality.

14. The video synchronization system according to claim 12 , wherein said video synchronization system is configured to decrease time intervals between occasions of modulating said alignment identifier if a value of said QoS parameter represents a better QoS than a defined QoS or if a value of said quality score represents a better quality than a defined minimum quality.

15. The video synchronization system according to claim 12 , wherein said video synchronization system is configured to generate an alignment identifier resulting in a reduction of a duration of said VLC signal if a value of said QoS parameter represents a worse QoS than a defined minimum QoS or if a value of said quality score represents a worse quality than a defined minimum quality.

16. The video synchronization system according to claim 5 , wherein
said video synchronization system is configured to obtain a quality score representative of a quality of video data of said bitstream for at least one user device of said multiple user devices; and
said video syn

Assignees

• Ericsson Telefon Ab L M

Inventors

• Back Heidi-Maria
• Mecklin Tomas
• Opsenica Miljenko
• Wang Le

Classifications

• H04N5/0675 Primary

  for mixing the synchronising signals with the picture signal or mutually (electrical gates H03K17/00) · CPC title

• H04N21/242

  Synchronisation processes, e.g. processing of PCR [Programme Clock References] {(arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18)} · CPC title

• H04B10/116 Primary

  Visible light communication · CPC title

• H04N21/8547

  involving timestamps for synchronizing content · CPC title

• H04B10/516

  Details of coding or modulation · CPC title