Audio output based on dynamic audio frame selection

Patent metadata

Field	Value
Publication number	US-2026086763-A1
Application number	US-202418893358-A
Country	US
Kind code	A1
Filing date	Sep 23, 2024
Priority date	Sep 23, 2024
Publication date	Mar 26, 2026
Grant date	—

Abstract

Systems and methods are provided to output audio data based on metadata associated with the audio data. A system can provide a buffer command to a plurality of computing devices. Based on providing the buffer command, the system can obtain, from the plurality of computing devices, a plurality of sets of audio data and a plurality of sets of metadata. The system can identify, for each time period of a plurality of time periods, a respective audio frame from the plurality of sets of audio data based on the plurality of sets of metadata. The system can generate a continuous audio stream based on identifying the respective audio frames and can route an output based on the continuous audio stream to a computing system.

First claim

What is claimed is:

1 . A system comprising:
data processing hardware; and memory in communication with the data processing hardware, the memory storing instructions that when executed on the data processing hardware cause the data processing hardware to:
identify a plurality of first computing devices located within a first environment;
provide a buffer command to the plurality of first computing devices, wherein each computing device of the plurality of first computing devices is configured to synchronously flush a respective buffer based on the buffer command;
obtain, from the plurality of first computing devices, a plurality of first sets of audio data and associated metadata based on the buffer command, wherein each first set of audio data of the plurality of first sets of audio data comprises a respective audio frame for each time period of a plurality of time periods;
dynamically select, for each time period of the plurality of time periods, a respective audio frame from the plurality of first sets of audio data based on the associated metadata;
determine a second set of audio data based on dynamically selecting, for each time period of the plurality of time periods, the respective audio frame, wherein the second set of audio data comprises at least one respective audio frame obtained from each first computing device of the plurality of first computing devices;
generate a continuous audio stream based on the second set of audio data;
perform one or more of noise suppression, automatic gain control, or echo cancellation to the continuous audio stream and subsequently generate an output; and
route the output to one or more second computing devices located within a second environment.

2 . The system of claim 1 , wherein execution of the instructions on the data processing hardware further causes the data processing hardware to:
determine an active speaker based on one or more of the plurality of first sets of audio data or a plurality of third sets of audio data obtained from the plurality of first computing devices; and route an identifier of the active speaker to the one or more second computing devices.

3 . The system of claim 1 , wherein each first computing device of the plurality of first computing devices comprises:
a respective microphone to obtain a respective first set of audio data.

4 . The system of claim 1 , wherein the associated metadata indicates one or more of a rate, a gap, or an amplitude associated with a respective audio frame of the plurality of first sets of audio data.

5 . A method comprising:
providing a buffer command to a first computing device and a second computing device to cause the first computing device to flush a first buffer based on the buffer command and to cause the second computing device to flush a second buffer based on the buffer command; obtaining, from the first computing device, a first set of audio frames and a first set of metadata subsequent to providing the buffer command to the first computing device, wherein the first set of audio frames comprises, for each time period of a plurality of time periods, a respective audio frame of the first set of audio frames; obtaining, from the second computing device, a second set of audio frames and a second set of metadata subsequent to providing the buffer command to the second computing device, wherein the second set of audio frames comprises, for each time period of the plurality of time periods, a respective audio frame of the second set of audio frames; identifying, for each time period of the plurality of time periods, a respective audio frame from the first set of audio frames and the second set of audio frames based on the first set of metadata and the second set of metadata; determining a third set of audio frames based on identifying, for each time period of the plurality of time periods, the respective audio frame from the first set of audio frames and the second set of audio frames, wherein the third set of audio frames comprises a particular audio frame of the first set of audio frames and a particular audio frame of the second set of audio frames; generating a continuous audio stream based on the third set of audio frames; and routing an output based on the continuous audio stream to one or more of the first computing device, the second computing device, or a third computing device.

6 . The method of claim 5 , further comprising:
comparing the first set of metadata and the second set of metadata, wherein identifying, for each time period of the plurality of time periods, the respective audio frame from the first set of audio frames and the second set of audio frames is based on comparing the first set of metadata and the second set of metadata.

7 . The method of claim 5 , further comprising:
comparing the first set of metadata and the second set of metadata; and determining a volume associated with a first audio frame of the first set of audio frames exceeds a volume associated with a second audio frame of the second set of audio frames based on comparing the first set of metadata and the second set of metadata, wherein identifying, for each time period of the plurality of time periods, the respective audio frame from the first set of audio frames and the second set of audio frames comprises identifying the first audio frame based on determining the volume associated with the first audio frame exceeds the volume associated with the second audio frame.

8 . The method of claim 5 , further comprising:
comparing the first set of metadata and the second set of metadata; determining a volume associated with a first audio frame of the first set of audio frames exceeds a volume associated with a second audio frame of the second set of audio frames based on comparing the first set of metadata and the second set of metadata; and determining a gap associated with the first audio frame satisfies a threshold, wherein identifying, for each time period of the plurality of time periods, the respective audio frame from the first set of audio frames and the second set of audio frames comprises identifying the second audio frame based on determining the gap satisfies the threshold.

9 . The method of claim 5 , wherein the first set of metadata is based on a shape of the first set of audio frames, and wherein the second set of metadata is based on a shape of the second set of audio frames.

10 . The method of claim 5 , wherein generating the continuous audio stream comprises generating the continuous audio stream using a multiplexer.

11 . The method of claim 5 , further comprising:
determining that the first set of audio frames and the second set of audio frames are aligned.

12 . The method of claim 5 , further comprising:
obtaining, from a fourth computing device, a fourth set of audio frames and a third set of metadata, wherein the fourth set of audio frames comprises, for each time period of the plurality of time periods, a respective audio frame of the second set of audio frames; and determining the fourth set of audio frames are misaligned with one or more of the first set of audio frames or the second set of audio frames, wherein generating the continuous audio stream is further based on determining the fourth set of audio frames are misaligned with the one or more of the first set of audio frames or the second set of audio frames.

13 . The method of claim 5 , further comprising:
determining an active speaker based on the first set of audio frames, the first set of metadata, the second set of audio frames, and the second set of metadata; and routing, to the third computing device, an identifier of the act

Assignees

• Amazon Tech Inc

Inventors

• Andress Mark
• Gupta Saurabh
• Watson Taylor Elliot

Classifications

• H03G3/3089

  Control of digital or coded signals · CPC title

• G06F16/683

  using metadata automatically derived from the content · CPC title

• G10K11/17823

  Reference signals, e.g. ambient acoustic environment · CPC title

• G06F3/165 Primary

  Management of the audio stream, e.g. setting of volume, audio stream path · CPC title