Elementary stream multiplexing method, multiplexing system, encoding or decoding method and apparatus

Patent metadata

Field	Value
Publication number	US-9866898-B2
Application number	US-201414468413-A
Country	US
Kind code	B2
Filing date	Aug 26, 2014
Priority date	Mar 29, 2012
Publication date	Jan 9, 2018
Grant date	Jan 9, 2018

Abstract

At the time of an encoding process, a video elementary stream is generated and output by capturing and encoding a video image signal at a time interval of a video frame synchronous with a timing of a video synchronization signal, and an audio elementary stream is generated by capturing and encoding an audio signal at each audio interval. An audio packetized elementary stream where one packet has a stream length corresponding to the time interval of the video frame is multiplexed with the audio elementary stream, and the stream is output.

First claim

What is claimed is:

1. An elementary stream multiplexing method for compressing and encoding a video image signal and an audio signal and for delivering the video image signal and the audio signal in a stream format, the method comprising:
generating and outputting a video elementary stream by capturing and encoding the video image signal at a time interval of a video frame synchronous with a timing of a video synchronization signal at the time of an encoding process;
generating an audio elementary stream by capturing and encoding the audio signal at each audio interval at the time of the encoding process;
multiplexing with the audio elementary stream an audio packetized elementary stream where one packet has a stream length corresponding to the time interval of the video frame, and outputting the stream at the time of the encoding process;
decoding the video image signal from the video elementary stream at the time of a decoding process;
demultiplexing the audio elementary stream at the audio interval from the audio packetized elementary stream at the time of the decoding process;
decoding the audio signal from the audio elementary stream at the time of the decoding process;
outputting the decoded video image signal and the decoded audio signal in synchronization with the video synchronization signal at the time of the decoding process;
outputting, at each capturing start timing of the audio signal, a difference value of the timing from the timing of the video synchronization signal at the time of the encoding process;
generating a dummy audio elementary stream having a stream length corresponding to the difference value and multiplexing the audio packetized elementary stream with the generated dummy audio elementary stream when the difference value is output, at the time of the encoding process;
outputting the difference value on the basis of a stream length of the dummy audio elementary stream when the audio packetized elementary stream is multiplexed with the dummy audio elementary stream at the time of the decoding process; and
outputting the audio signal, which is obtained by decoding the audio elementary stream demultiplexed subsequently to the dummy audio elementary stream, at a timing shifted from the timing of the video synchronization signal by the difference value when the difference value is output, at the time of the decoding process.

2. An elementary stream encoding method for compressing and encoding a video image signal and an audio signal in a stream format, the method comprising:
generating and outputting a video elementary stream by capturing and encoding the video image signal at a time interval of a video frame synchronous with a timing of a video synchronization signal;
generating an audio elementary stream by capturing the audio signal and encoding the audio signal at each audio interval;
multiplexing with the audio elementary stream an audio packetized elementary stream where one frame has a stream length corresponding to a time interval of the video frame, and outputting the stream;
outputting, at each capturing start timing of the audio signal, a difference value of the timing from the timing of the video synchronization signal; and
generating a dummy audio elementary stream having a stream length corresponding to the difference value and multiplexing the audio packetized elementary stream with the generated dummy audio elementary stream, when the difference value is output.

3. An elementary stream decoding method for decoding a video image signal and an audio signal, which are encoded in a stream format, the method comprising:
decoding the video image signal from a video elementary stream;
demultiplexing an audio elementary stream at each audio interval from an audio packetized elementary stream;
decoding the audio signal from the audio elementary stream; and
outputting the decoded video image signal and the decoded audio signal in synchronization with a timing of a video synchronization signal;
outputting a difference value on the basis of a stream length of a dummy audio elementary stream when the audio packetized elementary stream is multiplexed with the dummy audio elementary stream; and
outputting the audio signal, which is obtained by decoding the audio elementary stream demultiplexed subsequently to the dummy audio elementary stream, at a timing shifted from the timing of the video synchronization signal by the difference value, when the difference value is output.

4. An elementary stream multiplexing system that compresses and encodes a video image signal and an audio signal, and passes the video image signal and the audio signal in a stream format, the system comprising a processor, wherein
the processor generates and outputs a video elementary stream by capturing and encoding the video image signal at a time interval of a video frame synchronous with a timing of a video synchronization signal, generates an audio elementary stream by capturing and encoding the audio signal at each audio interval, multiplexes with the audio elementary stream an audio packetized elementary stream where one packet has a stream length corresponding to the time interval of the video frame and outputs the stream, decodes the video image signal from the video elementary stream, demultiplexes the audio elementary stream at the audio interval from the audio packetized elementary stream, decodes the audio signal from the audio elementary stream, outputs the decoded video image signal and the decoded audio signal in synchronization with the video synchronization signal,
outputs, at each capturing start timing of the audio signal, a difference value of the timing from the timing of the video synchronization signal, generates a dummy audio elementary stream having a stream length corresponding to the difference value and multiplexes the audio packetized elementary stream with the generated dummy audio elementary stream when the difference value is output, outputs the difference value on the basis of a stream length of the dummy audio elementary stream when the audio packetized elementary stream is multiplexed with the dummy audio elementary stream, and outputs the audio signal, which is obtained by decoding the audio elementary stream demultiplexed subsequently to the dummy audio elementary stream, at a timing shifted from the timing of the video synchronization signal by the difference value, when the difference value is output.

5. An elementary stream encoding apparatus for compressing and encoding a video image signal and an audio signal in a stream format, the apparatus comprising a processor, wherein
the processor generates and outputs a video elementary stream by capturing and encoding the video image signal at a time interval of a video frame synchronous with a timing of a video synchronization signal, generates an audio elementary stream by capturing the audio signal and encoding the audio signal at each audio interval, multiplexes with the audio elementary stream an audio packetized elementary stream where one packet has a stream length corresponding to the time interval of the video frame and outputs the stream,
outputs, at each capturing start timing of the audio signal, a difference value of the timing from the timing of the video synchronization signal, generates a dummy audio elementary stream having a stream length corresponding to the difference value, and multiplexes the audio packetized elementary stream with the generated dummy audio elementary stream when the difference value is output.

6. An elementary stream decoding apparatus for decoding a video image signal and an audio signal, which are encoded in a stream format, the apparatus comprising a processor, wherein
the processor decodes the video image signal from a video elementary stream, demultiplexe

Assignees

• Fujitsu Ltd

Inventors

• Yamashita Kazuhiro
• Yamaguchi Yousuke
• Kamito Takafumi
• Misuda Yasuo
• Tomita Yoshihiro
• Takabayashi Yosuke

Classifications

• H04N21/23605

  Creation or processing of packetized elementary streams [PES] · CPC title

• H04N21/4307

  Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen · CPC title

• H04N21/4343 Primary

  Extraction or processing of packetized elementary streams [PES] · CPC title

• H04N21/43079

  of additional data with content streams on multiple devices · CPC title