Digital twin for microphone array system

Patent metadata

Field	Value
Publication number	US-11854567-B2
Application number	US-202117451831-A
Country	US
Kind code	B2
Filing date	Oct 22, 2021
Priority date	Oct 22, 2021
Publication date	Dec 26, 2023
Grant date	Dec 26, 2023

Abstract

One example includes a digital twin of a microphone array. The digital twin acts as a digital copy of a physical microphone array. The digital array allows the microphone array to be analyzed, simulated and optimized. Further, the microphone array can be optimized for performing sound quality operations such as noise suppression and speech intelligibility.

First claim

What is claimed is:

1. A method, comprising:
receiving inputs into a digital twin, the inputs including sound information from a microphone array deployed in an environment;
performing sound quality operations based on the inputs by the digital twin, wherein the digital twin includes a digital copy of the microphone array;
generating outputs of the sound quality operations.

2. The method of claim 1 , further comprising presenting a visual representation of the outputs in a user interface of a device.

3. The method of claim 2 , wherein the device comprises an augmented reality or virtual reality device.

4. The method of claim 1 , wherein the inputs include background noise, speech from remote participants, speech from other users in the environment, environment acoustics parameters, movement trajectory, a sound wave propagation model, and array parameters.

5. The method of claim 4 , wherein the array parameters include a location of the microphone array, an array geometry, array steering, and radiation patterns.

6. The method of claim 1 , further comprising
generating a sound map that identifies locations and levels of noise sources and room impulse responses.

7. The method of claim 6 , further comprising generating, for the microphone array, optimized parameters including array pattern directivity index, spatial structure, filter weights, spatial resolution, speech indelibility, power consumption, perceptual evaluation of speech quality, short-time objective intelligibility, and frequency-weighted signal to noise ratio.

8. The method of claim 1 , further comprising receiving feedback from a user and performing the sound operations using the user feedback.

9. The method of claim 1 , further comprising performing a simulation by the digital twin.

10. A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
receiving inputs into a digital twin, the inputs including sound information from a microphone array;
performing sound quality operations based on the inputs by the digital twin, wherein the digital twin includes a digital copy of the microphone array;
generating outputs of the sound quality operations.

11. The non-transitory storage medium of claim 10 , further comprising presenting a visual representation of the outputs in a user interface of a device.

12. The non-transitory storage medium of claim 11 , wherein the device comprises an augmented reality or virtual reality device.

13. The non-transitory storage medium of claim 10 , wherein the inputs include background noise, speech from remote participants, room acoustics parameters, movement trajectory, a sound wave propagation model, and array parameters.

14. The non-transitory storage medium of claim 13 , wherein the array parameters include a location of the microphone array, an array geometry, array steering, and radiation patterns.

15. The non-transitory storage medium of claim 10 , further comprising
generating a sound map that identifies locations and levels of noise sources and room impulse responses.

16. The non-transitory storage medium of claim 10 , further comprising generating, for the microphone array, optimized parameters including array pattern directivity index, spatial structure, filter weights, spatial resolution, speech indelibility, power consumption, perceptual evaluation of speech quality, short-time objective intelligibility, and frequency-weighted signal to noise ratio.

17. The non-transitory storage medium of claim 10 , further comprising receiving feedback from a user and performing the sound operations using the user feedback.

18. The non-transitory storage medium of claim 10 , further comprising performing a simulation by the digital twin.

19. A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to implement a digital twin of a physical device, the digital twin comprising:
an input configured to receive noise source data, background noise, movement trajectory, array parameters, room acoustic parameters, and a sound propagation model;
functions including data retrieval functions, optimization functions, simulation functions, sensing functions, performance prediction functions, and remote monitoring functions;
an output configured to generate outputs including a sound map, real-time data visualization, optimized array patterns and parameters; speech intelligibility, signal to noise ratio, and sound quality.

20. The non-transitory storage medium of claim 19 , the digital twin further comprising a user interface configured to visualize at least some of the outputs.

Assignees

• Emc Ip Holding Co Llc

Inventors

• Sha Danqing
• Seibel Amy N
• Bruno Eric
• Jia Zhen

Classifications

• G10L21/0232 Primary

  Processing in the frequency domain · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• H04R3/005 Primary

  for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407) · CPC title

• G10L2021/02087

  the noise being separate speech, e.g. cocktail party · CPC title

• G10L2021/02166

  Microphone arrays; Beamforming · CPC title