Ultra-low latency audio over bluetooth

Patent metadata

Field	Value
Publication number	US-11259192-B2
Application number	US-201816146138-A
Country	US
Kind code	B2
Filing date	Sep 28, 2018
Priority date	Sep 29, 2017
Publication date	Feb 22, 2022
Grant date	Feb 22, 2022

Abstract

Methods and apparatus for communicating audio packets with ultra-low latency at high data rates from an audio source device to one or more audio output devices over a wireless personal area network (WPAN) connection, such as via a Bluetooth connection. Latency is reduced by using time-efficient audio coding and decoding, limited retransmissions, reduced time and frequency of acknowledgements, and by combining Bluetooth Classic (BTC) packets for downlink audio and downlink control with Bluetooth Low Energy (BTLE) packets for uplink control, uplink acknowledgements, and inter-device wireless communication. The number of retransmissions and packet concealments per frame cycle can be limited to an upper threshold number to satisfy a low latency requirement.

First claim

What is claimed is:

1. A method for low latency communication for a primary audio output device paired with an audio source device and with a secondary audio output device, the method comprising:
by the primary audio output device:
detecting an error in a downlink audio packet received from the audio source device in a first portion of a frame of a particular frame cycle of a sequence of periodically repeating frame cycles, each frame cycle comprising a set of frames that include downlink audio packets;
transmitting, to the audio source device, an uplink control packet during a second portion of the frame of the particular frame cycle, the uplink control packet providing an indication to request retransmission of audio data of the downlink audio packet;
receiving, from the audio source device, a second downlink audio packet in a subsequent frame of the frame cycle, the second downlink audio packet including retransmission of the audio data; and
when an upper threshold number of retransmissions per frame cycle is reached for the particular frame cycle, disallowing requests for retransmissions of previously transmitted downlink audio packets and newly transmitted downlink audio packets, while allowing reception of newly transmitted downlink audio packets without retransmission, for all frames of the particular frame cycle that follow the subsequent frame; and

re-allowing requests for retransmissions of downlink audio packets for a subsequent frame cycle that immediately follows the particular frame cycle.

2. The method of claim 1 , further comprising:
by the primary audio output device during a final frame of the particular frame cycle:
receiving an inter-device control packet from the secondary audio output device, the inter-device control packet including indications of user-initiated commands received via the second audio output device; and
transmitting a second uplink control packet to the audio source device, the second uplink control packet including information derived from the indications of the inter-device control packet.

3. The method of claim 2 , wherein the second uplink control packet further comprises one or more of: an acknowledgement for at least one downlink audio packet, a block acknowledgement for a set of two or more downlink audio packets, a negative acknowledgement for at least one lost downlink audio packet, or control data.

4. The method of claim 2 , wherein the second uplink control packet further comprises user commands received via an interface of the primary audio output device.

5. The method of claim 2 , wherein the inter-device control packet is formatted in accordance with a Bluetooth Low Energy (BTLE) wireless communication protocol or extension thereof.

6. The method of claim 1 , wherein:
the downlink audio packet received in the first portion of the frame is formatted in accordance with a Bluetooth Classic (BTC) wireless communication protocol or extension thereof; and
the uplink control packet transmitted in the second portion of the frame is formatted in accordance with a Bluetooth Low Energy (BTLE) wireless communication protocol or extension thereof.

7. The method of claim 1 , wherein:
lengths of the downlink audio packets occupy a time interval less than half of corresponding frames that include the downlink audio packets; and
downlink audio packets include at least 300 bytes of uncompressed audio data and up to 50 bytes of control data.

8. The method of claim 1 , wherein retransmission of audio data initially transmitted in a frame, when allowed, is restricted to occur in an immediately subsequent frame.

9. The method of claim 1 , wherein:
each frame of the particular frame cycle spans at most 1250 microseconds; and
the particular frame cycle spans at most 20 milliseconds.

10. The method of claim 1 , wherein the upper threshold number of retransmissions per frame cycle is increased or decreased adaptively by the audio source device to accommodate more or fewer retransmissions per frame cycle.

11. A primary audio output device configurable for low latency communication when paired with an audio source device and with a secondary audio output device, the primary audio output device comprising:
one or more processors communicatively coupled to a memory storing instructions that, when executed by the one or more processors, cause the primary audio output device to:
receive audio data in downlink audio packets over a particular frame cycle of a sequence of periodically repeating frame cycles, each frame cycle comprising a set of frames that include downlink audio packets and each downlink audio packet occupying less than half of a corresponding frame of the frame cycle;
detect an error a downlink audio packet received in a frame of the particular frame cycle;
transmit an uplink control packet during a second half of a frame that includes the downlink audio packet, the uplink control packet providing an indication to request retransmission of the audio data of the downlink audio packet;
receive a retransmission of the audio data of the downlink audio packet in a subsequent frame of the particular frame cycle; and

when an upper threshold number of retransmissions per frame cycle is reached for the particular frame cycle, disallow requests for retransmissions of previously transmitted downlink audio packets and newly transmitted downlink audio packets, while allowing reception of newly transmitted downlink audio packets without retransmission, during all subsequent frames of the particular frame cycle that follow the subsequent frame; and
re-allow requests for retransmission of downlink audio packets for a subsequent frame cycle that immediately follows the particular frame cycle.

12. The primary audio output device of claim 11 , wherein retransmission of audio data initially transmitted in a frame, when allowed, is restricted to occur in an immediately subsequent frame.

13. The primary audio output device of claim 11 , wherein execution of the instructions further causes the primary audio output device to:
during a final frame of the particular frame cycle:
receive an inter-device control packet from the second audio output device, the inter-device control packet including indications of user-initiated commands received via the second audio output device; and
transmit a second uplink control packet to the audio source device, the second uplink control packet including information derived from the indications of the inter-device control packet.

14. The primary audio output device of claim 13 , wherein the second uplink control packet further comprises one or more of: an acknowledgement for at least one downlink audio packet, a block acknowledgement for a set of two or more downlink audio packets, a negative acknowledgement for at least one lost downlink audio packet, or control data.

15. The primary audio output device of claim 13 , wherein the second uplink control packet further comprises user commands received via an interface of the primary audio output device.

16. The primary audio output device of claim 13 , wherein:
the downlink audio packets are formatted in accordance with a Bluetooth Classic (BTC) wireless communication protocol or extension thereof; and
the uplink control packet and the second uplink control packet are formatted in accordance with a Bluetooth Low Energy (BTLE) wireless communication protocol or extension thereof.

17. The primary audio output device of claim 11 , wherein:
the downlink audio packets include at least 300 bytes of uncompressed audio data and up to 50 bytes of control d

Assignees

• Apple Inc

Inventors

• Hariharan Sriram
• Paycher Alon
• Irony Ran
• Anantharaman Sreeraman
• Silfvast Robert D

Classifications

• H04L43/0847 Primary

  Transmission error · CPC title

• H04W84/18

  Self-organising networks, e.g. ad-hoc networks or sensor networks · CPC title

• H04R3/12

  for distributing signals to two or more loudspeakers {(specially adapted for hearing aids H04R25/407)} · CPC title

• H04W84/12

  WLAN [Wireless Local Area Networks] · CPC title

• H04W24/02 Primary

  Arrangements for optimising operational condition · CPC title