Video/audio synchronization apparatus and video/audio synchronization method

Patent metadata

Field	Value
Publication number	US-9489980-B2
Application number	US-201514599712-A
Country	US
Kind code	B2
Filing date	Jan 19, 2015
Priority date	Feb 20, 2014
Publication date	Nov 8, 2016
Grant date	Nov 8, 2016

Abstract

A video/audio synchronization apparatus includes: a decoder configured to decode a frame contained in encoded video data and determine playback time of the frame based on a first clock signal, and decode encoded audio data and determine playback time of an associated audio signal contained in the decoded audio data and to be played back simultaneously with the frame, based on a second clock signal different from the first clock signal; a silent period detector configured to detect a silent period from within the associated audio signal; and an audio timing adjuster configured to reduce a difference between the playback time of the frame and the playback time of the associated audio signal by skipping or repeating a silent period contained within a period of the associated audio signal.

First claim

What is claimed is:

1. A video/audio synchronization apparatus comprising:
a decoder configured to decode a frame contained in encoded video data and determine playback time of the frame based on a first clock signal, and decode encoded audio data and determine playback time of an associated audio signal contained in the decoded audio data and to be played back simultaneously with the frame, based on a second clock signal different from the first clock signal;
a silent period detector configured to detect a silent period from within the associated audio signal;
a delay time measure configured to measure a delay time representing how much the playback time of the associated audio signal determined based on the second clock signal is delayed with respect to the playback time of the frame determined based on the first clock signal; and
an audio timing adjuster configured to reduce a difference between the playback time of the frame and the playback time of the associated audio signal by skipping a silent period contained within a period of the associated audio signal when the delay time indicates that the playback time of the associated audio signal is later than the playback time of the frame, while by repeating the silent period when the delay time indicates that the playback time of the associated audio signal is earlier than the playback time of the frame.

2. The video/audio synchronization apparatus according to claim 1 , wherein:
the audio timing adjuster skips or repeats the silent period, depending on the delay time, until the delay time becomes shorter in length than a predetermined period or until there is no longer the silent period that can be skipped.

3. The video/audio synchronization apparatus according to claim 2 , wherein the predetermined period defines a minimum interval during which a silent period is to be skipped or repeated.

4. The video/audio synchronization apparatus according to claim 1 , wherein:
when the delay time indicates that the playback time of the associated audio signal is later than the playback time of the frame by an amount equal to or greater than a predetermined threshold value, the audio timing adjuster skips the silent period over a predetermined length of time shorter than an interval between two successive frames, and when the delay time indicates that the playback time of the associated audio signal is earlier than the playback time of the frame by an amount equal to or greater than the predetermined threshold value, the audio timing adjuster repeats the silent period over the predetermined length of time.

5. A video/audio synchronization method comprising:
decoding a frame contained in encoded video data and determining playback time of the frame based on a first clock signal;
decoding encoded audio data and determining playback time of an associated audio signal contained in the decoded audio data and to be played back simultaneously with the frame, based on a second clock signal different from the first clock signal;
detecting a silent period from within the associated audio signal;
measuring a delay time representing how much the playback time of the associated audio signal determined based on the second clock signal is delayed with respect to the playback time of the frame determined based on the first clock signal; and
reducing a difference between the playback time of the frame and the playback time of the associated audio signal by skipping a silent period contained within a period of the associated audio signal when the delay time indicates that the playback time of the associated audio signal is later than the playback time of the frame, while by repeating the silent period when the delay time indicates that the playback time of the associated audio signal is earlier than the playback time of the frame.

6. The video/audio synchronization method according to claim 5 , wherein:
the reducing the difference between the playback time of the frame and the playback time of the associated audio signal skips or repeats the silent period, depending on the delay time, until the delay time becomes shorter in length than a predetermined period or until there is no longer the silent period that can be skipped.

7. The video/audio synchronization method according to claim 6 , wherein the predetermined period defines a minimum interval during which a silent period is to be skipped or repeated.

8. The video/audio synchronization method according to claim 5 , wherein:
when the delay time indicates that the playback time of the associated audio signal is later than the playback time of the frame by an amount equal to or greater than a predetermined threshold value, the reducing the difference between the playback time of the frame and the playback time of the associated audio signal skips the silent period over a predetermined length of time shorter than an interval between two successive frames, and when the delay time indicates that the playback time of the associated audio signal is earlier than the playback time of the frame by an amount equal to or greater than the predetermined threshold value, the reducing the difference between the playback time of the frame and the playback time of the associated audio signal repeats the silent period over the predetermined length of time.

9. A non-transitory computer-readable recording medium having recorded thereon a computer program for video/audio synchronization that causes a computer to execute a process comprising:
decoding a frame contained in encoded video data and determining playback time of the frame based on a first clock signal;
decoding encoded audio data and determining playback time of an associated audio signal contained in the decoded audio data and to be played back simultaneously with the frame, based on a second clock signal different from the first clock signal;
detecting a silent period from within the associated audio signal;
measuring a delay time representing how much the playback time of the associated audio signal determined based on the second clock signal is delayed with respect to the playback time of the frame determined based on the first clock signal; and
reducing a difference between the playback time of the frame and the playback time of the associated audio signal by skipping a silent period contained within a period of the associated audio signal when the delay time indicates that the playback time of the associated audio signal is later than the playback time of the frame, while by repeating the silent period when the delay time indicates that the playback time of the associated audio signal is earlier than the playback time of the frame.

Assignees

• Fujitsu Ltd

Inventors

• Takabayashi Yosuke
• Umezaki Yasushi
• Misuda Yasuo
• Ishikawa Shotaro

Classifications

• G11B27/007 Primary

  reproducing continuously a part of the information, i.e. repeating · CPC title

• G11B27/3081

  used signal is a video-frame or a video-field (P.I.P) · CPC title

• G11B27/3027

  used signal is digitally coded · CPC title

• H04N5/04 Primary

  Synchronising (for television systems using pulse code modulation H04N7/56) · CPC title

• H04N21/439

  Processing of audio elementary streams · CPC title