Methods and devices for controlling speech quality

Patent metadata

Field	Value
Publication number	US-2016286575-A1
Application number	US-201615007338-A
Country	US
Kind code	A1
Filing date	Jan 27, 2016
Priority date	Mar 24, 2015
Publication date	Sep 29, 2016
Grant date	—

Abstract

The disclosure relates to methods and devices for controlling speech quality, in particular by controlling end-to-end latency and by improving speech quality in case of mobility scenarios. A method 200 for controlling end-to-end latency between receiving and processing audio frames includes: receiving 201 a data packet comprising at least one coded audio frame; storing 202 the received data packet in a packet buffer; retrieving 203 the received data packet from the packet buffer and decoding the at least one coded audio frame into audio samples; and processing 204 the audio samples, wherein a scheduling of retrieving 203 the received data packet from the packet buffer and decoding the at least one coded audio frame is based on a target criterion with respect to audio quality of the audio samples and latency between receiving the data packet and processing the audio samples, and wherein the scheduling is dynamically and smoothly shifted in time in order to avoid audio distortions.

First claim

1 . A method for controlling end-to-end latency between receiving and processing audio frames, the method comprising:
receiving a data packet comprising at least one coded audio frame; storing the received data packet in a packet buffer; retrieving the received data packet from the packet buffer and decoding the at least one coded audio frame into audio samples; and processing the audio samples, wherein a scheduling of retrieving the received data packet from the packet buffer and decoding the at least one coded audio frame is based on a target criterion with respect to audio quality of the audio samples and latency between receiving the data packet and processing the audio samples, and wherein the scheduling is dynamically and smoothly shifted in time in order to avoid audio distortions.

2 . The method of claim 1 ,
wherein scheduling of retrieving the received data packet from the packet buffer and decoding the at least one coded audio frame is based on time-scaling the at least one decoded audio frame in order to move a processing of the received data packet in an optimal position with respect to a minimal latency.

3 . The method of claim 2 ,
wherein time-scaling the at least one decoded audio frame comprises at least one of: frame compression for reducing the latency between receiving the data packet and processing the audio samples, frame expansion for increasing the latency between receiving the data packet and processing the audio samples, and idle activity with respect to frame compression and frame expansion for keeping the latency between receiving the data packet and processing the audio samples.

4 . The method of claim 2 ,
wherein the time-scaling is used for shifting the scheduling in order to modify a position of the at least one decoded audio frame.

5 . The method of claim 2 ,
wherein time-scaling is based on one of speech frame compression and speech frame expansion.

6 . The method of claim 1 , comprising:
determining an optimal scheduling based on at least one of the following information: statistics about latency between received data and played-out data, modem statistics based on at least one of reception rate and retransmission rate.

7 . The method of claim 1 ,
wherein the data packet is part of a stream of asynchronously received data packets.

8 . The method of claim 7 , further comprising:
determining statistics of the latency between receiving a respective data packet of the stream of data packets and processing the audio samples of the respective data packet; and using the statistics for the scheduling.

9 . The method of claim 1 ,
wherein retrieving the received data packet from the packet buffer and decoding the at least one coded audio frame and processing the audio samples is based on a master clock.

10 . The method of claim 1 ,
wherein scheduling of retrieving the received data packet from the packet buffer and decoding the at least one coded audio frame is performed by adjusting at least one of: clocking of retrieving the received data packet from the packet buffer and decoding the at least one coded audio frame, and clocking of processing the audio samples.

11 . The method of claim 1 , further comprising:
determining a latency between the received data packet and the processed audio samples; determining an audio quality of the audio samples; and scheduling retrieving the received data packet from the packet buffer and decoding the at least one coded audio frame such that the audio quality is above a first threshold and the latency between receiving the data packet and processing the audio samples is below a second threshold.

12 . The method of claim 1 ,
wherein processing the audio samples comprises: initializing a playout time of the audio samples based on at least one of the following: a position indicating a start of the audio frame, a number of retransmissions of the audio frame, a retransmission of the data packet comprising the audio frame, an internal processing time.

13 . A device for controlling end-to-end latency between receiving and processing audio frames, the device comprising:
a packet buffer configured to receive a data packet comprising at least one coded audio frame; a decoder configured to retrieve the received data packet from the packet buffer and to decode the at least one coded audio frame into audio samples; an audio processor configured to process the audio samples; and a scheduler configured to schedule retrieving the received data packet from the packet buffer and decoding of the at least one coded audio frame based on a target criterion with respect to audio quality of the audio samples and latency between receiving the data packet by the packet buffer and processing the audio samples by the audio processor, wherein the scheduler is configured to dynamically and smoothly shift the scheduling in time in order to avoid audio distortions.

14 . The device of claim 13 , further comprising:
an audio buffer coupled between the decoder and the audio processor, wherein the decoder is configured to store the audio samples in the audio buffer and the audio processor is configured to retrieve the audio samples from the audio buffer.

15 . The device of claim 14 ,
wherein the scheduler is configured to adjust at least one of: an access rate of the decoder for storing the audio samples in the audio buffer, an access rate of the audio processor for retrieving the audio samples from the audio buffer, an access rate of pull requests to the audio buffer.

16 . The device of claim 15 ,
wherein the scheduler is configured to adjust the access rate of the decoder based on a first clock and the access rate of the audio processor based on a second clock, wherein the first clock and the second clock are derived from a master clock or any other synchronization mechanism.

17 . The device of claim 13 ,
wherein the scheduler is configured to schedule retrieving the received data packet from the packet buffer and decoding of the at least one coded audio frame based on adjusting a time-scaling of the decoder for decoding the at least one coded audio frame.

18 . The device of claim 13 ,
wherein the decoder comprises at least one of a speech decoder and a speech time scaler.

19 . A method for adjusting a size of a jitter buffer in a media processing circuit of a mobile terminal, the method comprising:
depacketizing at least one coded media frame from a received radio signal; storing the depacketized at least one coded media frame in a jitter buffer; retrieving the at least one coded media frame from the jitter buffer and decoding the at least one coded media frame into media samples; determining a jitter model based on information indicating a mobility state of the mobile terminal; adjusting the jitter model based on a history of the information indicating the mobility state; and adjusting a size of the jitter buffer based on the jitter model.

20 . The method of claim 19 ,
wherein the information indicating the mobility state of the mobile terminal comprises at least one of the following information related to the mobile terminal: speed or velocity information, location information, environment information, time information, change of velocity or acceleration information.

21 . The method of claim 19 , further comprising:
adjusting the size of the jitter buffer based on a network jitter estimated based on the information indicating the mobility stat

Assignees

• Intel Corp

Inventors

• Weingertner Philippe
• Serre Eric
• Beaugeant Christophe
• Parron Jerome

Classifications

• H04L47/28

  in relation to timing considerations · CPC title

• H04L47/56

  implementing delay-aware scheduling · CPC title

• G10L19/167

  Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes · CPC title

• H04M1/2535

  adapted for voice communication over an Internet Protocol [IP] network (Voice over Internet Protocol (VoIP) network equipment and services H04M7/006; implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP] H04L69/16) · CPC title

• H04W4/027 Primary

  using movement velocity, acceleration information · CPC title