Speech processing optimizations based on microphone array

Patent metadata

Field	Value
Publication number	US-12387727-B1
Application number	US-202418439412-A
Country	US
Kind code	B1
Filing date	Feb 12, 2024
Priority date	Mar 21, 2018
Publication date	Aug 12, 2025
Grant date	Aug 12, 2025

Abstract

Systems and methods for utilizing microphone array information for acoustic modeling are disclosed. Audio data may be received from a device having a microphone array configuration. Microphone configuration data may also be received that indicates the configuration of the microphone array. The microphone configuration data may be utilized as an input vector to an acoustic model, along with the audio data, to generate phoneme data. Additionally, the microphone configuration data may be utilized to train and/or generate acoustic models, select an acoustic model to perform speech recognition with, and/or to improve trigger sound detection.

First claim

What is claimed is:

1. A method comprising:
receiving first input data from a first device having a first device location of a building;
determining first contextual data indicating one or more first physical objects disposed in association with the first device location;
inputting, into a first model, the first input data and the first contextual data;
generating, using the first model and based at least in part on the first input data and the first contextual data, first output data;
determining that the first device is in a second device location of the building, wherein the second device location is associated with second contextual data indicating one or more second physical objects that differ from the one or more first physical objects;
generating, using the first model and based at least in part on second input data and the second contextual data, second output data that differs at least in part from the first output data, the second output data indicating an action to be performed by the first device responsive to a user query, wherein the action includes controlling a playback parameter of an media output of the first device associated with the user query; and
sending, based at least in part on generating the second output data, a command to the first device that causes the first device to perform the action.

2. The method of claim 1 , wherein generating the first output data comprises generating text data based at least in part on the first input data and the first contextual data.

3. The method of claim 1 , wherein the first contextual data indicates a distance between the first device and a second device of the building.

4. The method of claim 3 , wherein generating the first output data using the first model comprises generating the first output data using the first model with the distance as an input to the first model.

5. The method of claim 1 , wherein the first input data indicates the first device location with respect to a source of the first contextual data.

6. The method of claim 1 , further comprising:
identifying additional input data;
generating third contextual data based at least in part on the additional input data; and
wherein generating the second output data comprises generating the second output data with the third contextual data as input to the first model.

7. The method of claim 1 , further comprising:
determining an environmental context associated with at least a portion of the first input data; and
wherein generating the first output data comprises generating the first output data with the environmental context as input to the first model.

8. The method of claim 1 , further comprising:
determining that a distance between the first device and a second device in the building satisfies a threshold distance; and
wherein generating the first output data comprises generating the first output data based at least in part on the distance satisfying the threshold distance.

9. The method of claim 1 , further comprising determining characteristics of the second device location, wherein the second contextual data is based at least in part on the characteristics of the second device location.

10. The method of claim 1 , wherein the playback parameter includes one or more of mute, volume control, or play.

11. A system comprising:
one or more processors; and
non-transitory computer-readable media storing instructions that, when executed by on the one or more processors, cause the one or more processors to perform operations comprising:
receiving first input data from a first device having a first device location of a building;
determining first contextual data indicating one or more first physical objects disposed in association with the first device location;
inputting, into a first model, the first input data and the first contextual data;
generating, using the first model and based at least in part on the first input data and the first contextual data, first output data;
determining that the first device is in a second device location of the building, wherein the second device location is associated with second contextual data indicating one or more second physical objects that differ from the one or more first physical objects;
generating, using the first model and based at least in part on second input data and the second contextual data, second output data that differs at least in part from the first output data, the second output data indicating an action to be performed by the first device responsive to a user query, wherein the action includes instructions controlling a playback parameter of a media output of the first device that is associated with the user query; and
sending, based at least in part on generating the second output data, a command to the first device that causes the first device to perform the action.

12. The system of claim 11 , wherein generating the first output data comprises generating text data based at least in part on the first input data and the first contextual data.

13. The system of claim 11 , wherein the first contextual data indicates a distance between the first device and a second device of the building.

14. The system of claim 13 , wherein generating the first output data using the first model comprises generating the first output data using the first model with the distance as an input to the first model.

15. The system of claim 11 , wherein the first input data indicates the first device location with respect to a source of the first contextual data.

16. The system of claim 11 , the operations further comprising:
identifying additional input data;
generating third contextual data based at least in part on the additional input data; and
wherein generating the second output data comprises generating the second output data with the third contextual data as input to the first model.

17. The system of claim 11 , the operations further comprising:
determining an environmental context associated with at least a portion of the first input data; and
wherein generating the first output data comprises generating the first output data with the environmental context as input to the first model.

18. The system of claim 11 , the operations further comprising:
determining that a distance between the first device and a second device in the building satisfies a threshold distance; and
wherein generating the first output data comprises generating the first output data based at least in part on the distance satisfying the threshold distance.

19. The system of claim 11 , the operations further comprising determining characteristics of the second device location, wherein the second contextual data is based at least in part on the characteristics of the second device location.

20. The system of claim 11 , wherein the playback parameter includes one or more of mute, volume control, or play.

Assignees

• Amazon Tech Inc

Inventors

• Sundaram Shiva Kumar
• Wu Minhua
• Raju Anirudh
• Matsoukas Spyridon
• Mandal Arindam
• Kumatani Kenichi

Classifications

• G06F40/40

  Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) · CPC title

• G10L21/0208

  Noise filtering · CPC title

• G10L2015/088

  Word spotting · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L2021/02166

  Microphone arrays; Beamforming · CPC title