Error resilient tools for audio encoding/decoding

Patent metadata

Field	Value
Publication number	US-2025316282-A1
Application number	US-202519243429-A
Country	US
Kind code	A1
Filing date	Jun 19, 2025
Priority date	Dec 23, 2022
Publication date	Oct 9, 2025
Grant date	—

Abstract

There are provided examples of audio signal representation encoders, audio encoders, audio signal representation decoders, and audio decoders, in particular using error resilient tools, e.g. for learnable applications.In one examples, there is provided an audio signal representation decoder configured to decode an audio signal representation from a bitstream, the bitstream being divided in a sequence of packets, the audio signal representation decoder comprising:a bitstream reader, configured to sequentially read the sequence of packets;a packet loss controller, configured to check whether a current packet is well received or is to be considered as lost;a quantization index converter, configured, in case the packet loss controller has determined that the current packet is well received, to convert at least one index extracted from the current packet onto at least one current code from at least one codebook, thereby forming at least one portion of the audio signal representation; andwherein the audio signal representation decoder is configured, in case the packet loss controller has determined that the current packet is to be considered as lost, to generate, through at least one learnable predictor layer, at least one current code by prediction from at least one preceding code or index, thereby forming at least one portion of the audio signal representation.

First claim

1 . An encoder, comprising:
an audio signal representation generator configured to generate, through at least one learnable layer, an audio signal representation as a representation of an audio signal, the audio signal representation comprising a sequence of tensors; a quantizer configured to convert each current tensor of the sequence of tensors onto at least one index, wherein each index is obtained from at least one codebook associating a plurality of tensors to a plurality of indexes; a bitstream writer configured to write packets in the bitstream, so that a current packet comprises the at least one index for the current tensor of the sequence of tensors, wherein the encoder is configured to write redundancy information of the current tensor in at least one preceding or following packet of the bitstream different from the current packet and/or to write, in the current packet, redundancy information of a tensor, different from the current tensor, in the current packet.

2 . The encoder of claim 1 , wherein the at least one codebook associates parts of tensors to indexes, so that the quantizer converts the current tensor onto a plurality of indexes.

3 . The encoder of claim 1 , wherein the at least one codebook comprises:
a base codebook associating main portions of tensors to indexes; and at least one low-ranking codebook associating residual portions of tensors to indexes, wherein the at least one current tensor has at least one main portion and at least one residual portion, wherein the quantizer is configured to convert the main portion of the at least one current tensor onto at least one high-ranking index, and the at least one residual portion of the at least one tensor onto at least one low-ranking index, so that the bitstream writer writes, in the bitstream, both the high-ranking index and the at least one low-ranking index.

4 . The encoder of claim 3 , configured to provide the redundancy information with at least the high-ranking index(es) of the at least one preceding or following packet, but not at least the lowest-ranking low-ranking index(es) of the same at least one preceding or following packet.

5 . The encoder of claim 1 , configured to split the current tensor into a plurality of subtensors, so as to quantize each subtensor.

6 . The encoder of claim 1 , configured to decompose the current tensor among a main portion and at least one residual portion, so as to quantize the main portion and the at least one residual portion.

7 . The encoder of claim 1 , configured to transmit the bitstream to a receiver through a communication channel.

8 . The encoder of claim 7 , configured to monitor the payload state of the communication channel, so as, in case the payload state in the communication channel is over a predetermined threshold, to increase the quantity of redundancy information.

9 . The encoder of claim 3 , configured to transmit the bitstream to a receiver through a communication channel
and further configured to monitor the payload state of the communication channel, so as, in case the payload state in the communication channel is over a predetermined threshold, to increase the quantity of redundancy information and further configured: in case the payload in the communication channel is below the predetermined threshold, to only transmit, as redundancy information, for each current packet, high-ranking indexes of the at least one preceding or following packet; and/or in case the payload of the communication channel is over the predetermined threshold, to transmit, as redundancy information, for each current packet, both the high-ranking indexes of the at least one preceding or following packet and at least some low-ranking indexes of the same at least one preceding or following packet.

10 . The encoder of claim 8 , configured to compute a packet offset between the current packet and the at least one preceding or following packet having the redundant information at least in function of the payload of the communication channel.

11 . The encoder of claim 8 , configured to compute a packet offset between the current packet and the at least one preceding or following packet having the redundant information at least in function of the envisioned application.

12 . The encoder of claim 8 , configured to compute a packet offset between the current packet and the at least one preceding or following packet having the redundant information at least in function of an input provided by the end-user.

13 . The encoder of claim 9 , configured to compute a packet offset between the current packet and the at least one preceding or following packet having the redundant information at least in function of the payload of the communication channel, in such a way that the higher the payload in the communication channel, or the higher the error rate in the communication channel, the higher the packet offset.

14 . The encoder of claim 8 , wherein the at least one codebook comprises a redundancy codebook associating a plurality of tensors to a plurality of indexes, wherein the encoder is configured to write the redundancy information of the current tensor in the at least one preceding or following packet of the bitstream different from the current packet as an index received from the at least one quantization codebook.

15 . A method comprising:
generating, through at least one learnable layer, an audio signal representation as a representation of an audio signal, the audio signal representation comprising a sequence of tensors; converting each current tensor of the sequence of tensors onto at least one index, wherein each index is obtained from at least one codebook associating a plurality of tensors to a plurality of indexes; writing packets in a bitstream, so that a current packet comprises the at least one index for the current tensor of the sequence of tensors, wherein the method comprises writing redundancy information of the current tensor in at least one preceding or following packet of the bitstream different from the current packet, and/or writing, in the current packet, redundancy information of at least one tensor to be written in at least one preceding or following packet of the bitstream different from the current packet.

16 . A non-transitory digital storage medium having a computer program stored thereon to perform the method comprising:
generating, through at least one learnable layer, an audio signal representation as a representation of an audio signal, the audio signal representation comprising a sequence of tensors; converting each current tensor of the sequence of tensors onto at least one index, wherein each index is obtained from at least one codebook associating a plurality of tensors to a plurality of indexes; writing packets in a bitstream, so that a current packet comprises the at least one index for the current tensor of the sequence of tensors,
wherein the method comprises writing redundancy information of the current tensor in at least one preceding or following packet of the bitstream different from the current packet, and/or writing, in the current packet, redundancy information of at least one tensor to be written in at least one preceding or following packet of the bitstream different from the current packet,
when said computer program is run by a computer.

Assignees

• Fraunhofer Ges Forschung

Inventors

• Gupta Kishan
• Pia Nicola
• Korse Srikanth
• Fuchs Guillaume
• Multrus Markus
• Schnell Markus
• Brendel Andreas

Classifications

• G06N3/094

  Adversarial learning · CPC title

• G10L2019/0001

  Codebooks · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title