Timestamp-based audio and video processing method and system thereof

Patent metadata

Field	Value
Publication number	US-9602769-B2
Application number	US-201614993091-A
Country	US
Kind code	B2
Filing date	Jan 12, 2016
Priority date	Jan 12, 2015
Publication date	Mar 21, 2017
Grant date	Mar 21, 2017

Abstract

A timestamp-based audio and video processing method and a system thereof are provided. The method is adapted to an electronic device having an image capturing device and an audio input device and includes the following steps. First, an image signal and an audio signal are extracted within a same time interval by respectively using the image capturing device and the audio input device so as to generate video data and audio data, and the time interval is converted to timestamp data, where the video data includes a single video frame with a variable file size, the audio data includes multiple audio files with a fixed file size. Next, the video data, the audio data, and the timestamp data are integrated into transmission data.

First claim

What is claimed is:

1. A timestamp-based audio and video processing method, adapted to an image capturing device and an audio receiving device, comprising:
obtaining first transmission data from a first source and obtaining second transmission data from a second source, wherein the first transmission data is an audio-video packet comprising first video data, first audio data, and first timestamp data, wherein the first video data comprises a single video frame with a variable file size, wherein the first audio data comprises a plurality of audio files with a fixed file size, wherein the second transmission data is a second audio-video packet comprising second video data, second audio data, and second timestamp data, wherein the second video data comprises a single video frame with a variable file size, wherein the second audio data comprises a plurality of audio files with a fixed file size, and wherein the electronic device comprises a first data buffer having a plurality of first temporary positions and a second data buffer having a plurality of second temporary positions;
pushing the first audio-video packet to the first temporary positions sequentially according to the first timestamp data in the first audio-video packet and pushing the second audio-video packet to the second temporary positions sequentially according to the second timestamp data in the second audio-video packet; and
when the first audio-video packet is sorted at first among the first temporary positions and when the second audio-video packet is sorted at first among the second temporary positions, concurrently playing the first video data and the first audio data in the first audio-video packet as well as the second video data and the second audio data in the second audio-video packet respectively by using the screen and the audio playing device according to the first timestamp data of the first transmission data and the second timestamp data of the second transmission data.

2. The method according to claim 1 , wherein the electronic device concurrently plays the first video data and the second video data by a multi-window or a pop-up play on the screen.

3. A timestamp-based audio and video processing method, adapted to an image capturing device and an audio receiving device, comprising:
obtaining first transmission data from a first source, wherein the first transmission data is a first video packet comprising first video data comprising first video data and a first audio packet comprising first audio data and first timestamp data, wherein the first video data comprises a single video frame with a variable file size, wherein the first audio data comprises a plurality of audio files with a fixed file size, and wherein the electronic device comprises a first video buffer having a single first video temporary position and a first audio buffer having a plurality of first audio temporary positions;
pushing the first video packet to the first video temporary position;
pushing the first audio packet to the first audio temporary positions sequentially according to the first timestamp data in the first audio packet; and
playing the first video data and the first audio data respectively by using the screen and the audio playing device according to the first timestamp data of the first transmission data.

4. The method according to claim 3 , wherein the step of pushing the first video packet to the first video temporary position comprises:
when there exists another packet at the first video temporary position, overwriting the another packet by the first video packet.

5. The method according to claim 3 , wherein the step of playing the first video data and the first audio data respectively by using the screen and the audio playing device according to the first timestamp data of the first transmission data comprises:
immediately playing the first video data in the first video packet at the first video temporary position by using the screen; and
when the first audio packet is sorted first among the first audio temporary positions, playing the first audio data in the first audio packet by using the audio playing device.

6. The method according to claim 3 , wherein the electronic device further comprises a second video buffer having a single second video temporary position and a second audio buffer having a plurality of second audio temporary positions, and wherein the method further comprises:
obtaining second transmission data from a second source, wherein the second transmission data is a second video packet comprising second video data and a second audio packet comprising second audio data and second timestamp data, wherein the second video data comprises a single video frame with a variable file size, and wherein the second audio data comprises a plurality of audio files with a fixed file size;
pushing the second video packet to the second video temporary position;
pushing the second audio packet to the second audio temporary positions sequentially according to the second timestamp data in the second audio packet;
selecting one of the first video data in the first video packet and the second video data in the second video packet to be selected video data;
when the first audio packet and the second audio packet are respectively sorted at first among the first audio temporary positions and the second audio temporary positions, integrating the first audio data in the first audio packet and the second audio data in the second audio packet into integrated audio data; and
playing the selected video data and the integrated audio data by respectively using the screen and the audio playing device.

7. The method according to claim 6 , wherein the step of selecting one of the first video data in the first video packet and the second video data in the second video packet to be the selected video data comprises:
calculating an average volume of the first audio data and an average volume of the second audio data respectively to obtain a first average volume and a second average volume;
when the first average volume is greater than the second average volume, selecting the first video data to be the selected video data; and
when the second average volume is greater than the first average volume, selecting the second video data to be the selected video data.

8. A timestamp-based audio and video processing system comprising:
at least one client device, wherein a first client device and a second client device of the at least one client device are configured to respectively generate first client transmission data and second client transmission data, wherein the first client transmission data is a first audio-video packet comprising first client video data, first client audio data, and first client timestamp data, wherein the second client transmission data is a second audio-video packet comprising second client video data, second client audio data, and second client timestamp data, wherein the first client video data comprises a single video frame with a variable file size, wherein the first client audio data comprises a plurality of audio files with a fixed file size, wherein the second client video data comprises a single video frame with a variable file size, and wherein the second client audio data comprises a plurality of audio files with a fixed file size; and
a host device, connected to each of the at least one client device and comprising a first data buffer having a plurality of first temporary positions and a second data buffer having a plurality of second temporary positions,
wherein the host device receives the first client transmission data and the second client transmission data, pushes the first audio-video packet to the first temporary positions sequentially according to the first client

Assignees

• Cheng Hsin-Chieh
• Chang Kun-Hsuan
• Weng Ming-Che
• Compal Electronics Inc

Inventors

• Cheng Hsin-Chieh
• Chang Kun-Hsuan
• Weng Ming-Che

Classifications

• H04N23/66

  Remote control of cameras or camera parts, e.g. by remote control devices · CPC title

• H04N21/23602 Primary

  Multiplexing isochronously with the video sync, e.g. according to bit-parallel or bit-serial interface formats, as SDI · CPC title

• H04N7/142

  Constructional details of the terminal equipment, e.g. arrangements of the camera and the display · CPC title

• H04L67/025

  for remote control or remote monitoring of applications · CPC title

• H04N21/8547

  involving timestamps for synchronizing content · CPC title