Systems and methods for identifying an acoustic source based on observed sound

Patent metadata

Field	Value
Publication number	US-11568731-B2
Application number	US-202016872168-A
Country	US
Kind code	B2
Filing date	May 11, 2020
Priority date	Jul 15, 2019
Publication date	Jan 31, 2023
Grant date	Jan 31, 2023

Abstract

An electronic device includes a processor, and a memory containing instructions that, when executed by the processor, cause the electronic device to learn a sound emitted by a legacy device and to issue an output when the electronic device subsequently hears the sound. For example, the electronic device can receive a training input and extract a compact representation of a sound in the training input, which the device stores. The device can receive an audio signal corresponding to an observed acoustic scene and extract a representation of the observed acoustic scene from the audio signal. The electronic device can determine whether the sound is present in the observed acoustic scene at least in part from a comparison of the representation of the observed acoustic scene with the representation of the sound. The electronic device emits a selected output responsive to determining that the sound is present in the acoustic scene.

First claim

We currently claim:

1. An electronic device comprising a microphone, a processor, and a memory containing instructions that, when executed by the processor, cause the electronic device to:
receive a first audio signal corresponding to an input to the microphone, wherein the input comprises a reference version of a sound;
establish, based on the received first audio signal, a selected threshold level;
from the first audio signal, extract a representation of the sound in the input, wherein the representation of the sound is a reference representation of the sound, and the reference representation of the sound comprises reverberation or background impairments below the selected threshold level;
store the representation of the sound;
receive a second audio signal corresponding to an acoustic scene observed by the microphone;
extract a representation of the observed acoustic scene from the second audio signal;
determine whether the sound is present in the observed acoustic scene at least in part from a comparison of the representation of the observed acoustic scene with the representation of the sound;
emit a selected output responsive to determining that the sound is present in the acoustic scene; and
update the stored representation of the sound when the processor determines the sound is present in an observed acoustic scene.

2. The electronic device according to claim 1 , wherein the instructions, when executed by the processor, further cause the electronic device to receive a further audio signal corresponding to the sound and to update the stored representation of the sound in correspondence with the further audio signal.

3. The electronic device according to claim 1 , wherein the reference representation of the sound corresponds to a combination of the reference version of the sound and one or more of a frequency response representative of an environment in which the electronic device operates, a background noise, or a combination thereof.

4. The electronic device according to claim 1 , wherein the reference representation of the sound comprises information pertaining to a direction from which the sound originates.

5. The electronic device according to claim 1 , wherein the instructions, when executed by the processor, further cause the electronic device to receive a plurality of other audio signals, each corresponding to a respective acoustic scene, and to define a second reference representation of the sound corresponding to each respective acoustic scene, and wherein each respective reference representation of the sound corresponds to a combination of the reference version of the sound with the respective other audio signal corresponding to the respective acoustic scene.

6. The electronic device according to claim 5 , wherein the instructions further cause the electronic device to communicate a classification of the sound to another electronic device or in a user-perceptible manner to a user, or both.

7. The electronic device according to claim 1 , wherein the instructions further cause the electronic device to request from a user authorization to extract the representation of the sound in the input.

8. The electronic device according to claim 1 , wherein the instructions further cause the electronic device to assign the representation of the sound to a selected class of device, and wherein the selected output contains information corresponding to the selected class of device.

9. The electronic device according to claim 1 , wherein the selected output comprises one or more of a visual output, a tactile output, an auditory output, an olfactory output, and a proprioceptive output a user-perceptible output or an output signal transmitted to another device.

10. An integrated circuit comprising circuitry configured to:
learn a sound emitted by another device when the sound recurs in an acoustic scene observed by a microphone;
establish, based on the learned sound, a selected threshold level;
after learning the sound, listen for and detect a presence of the sound in a sound field;
cause a memory to store a reference representation of the sound, wherein the reference representation includes an acoustic impairment below the selected threshold level;
responsive to a detected presence of the sound in the sound, emit an output; and
update the stored reference representation of the sound when the sound is detected in the sound field.

11. The integrated circuit according to claim 10 , wherein the output comprises a user-perceptible visual output, tactile output, auditory output, olfactory output, or proprioceptive output.

12. The integrated circuit according to claim 10 , wherein the circuitry is further configured to condition one or more acts including one or more of learning the sound, listening for the sound, and detecting the presence of the sound on receiving an input indicative of a user's authorization to perform the one or more acts.

13. The integrated circuit according to claim 10 , wherein the other device is an analog device and the output contains information that indicates the analog device emitted the sound.

14. The integrated circuit according to claim 10 , wherein the circuitry is further configured to listen for the sound combined with one or more other sounds corresponding to a selected acoustic scene.

15. The integrated circuit according to claim 10 , wherein the circuitry is further configured to discern a source of the learned sound according to a direction from which the learned sound emanates.

16. A method, comprising:
defining a reference representation of sound captured by a microphone of another device;
establishing, based on the sound, a selected threshold level, wherein the reference representation of the sound comprises at least one of a reverberation or an acoustic impairment below the selected threshold level;
extracting a representation of an acoustic scene;
comparing the representation of the acoustic scene with the reference representation of sound from the other device;
from the comparing, determining whether sound from the other device is present in the acoustic scene; and
responsive to determining sound from the other device is present, emitting a selected output.

17. The method according to claim 16 , wherein the selected output is a user-perceptible output.

18. The method according to claim 16 , wherein the other device is a first device, the reference representation is a first reference representation corresponding to the first device, and the method further comprises:
defining a second reference representation of a second sound received from a second device;
determining whether the second sound from the second device is present in the observed acoustic scene from a comparison of the representation of the observed acoustic scene with the second reference representation; and
responsive to determining the second sound from the second device is present, emitting a second selected output corresponding to a presence of the second sound from the second device.

19. The method according to claim 16 wherein the acoustic scene is a first acoustic scene, and the method further comprises:
extracting a representation of a second acoustic scene; and
determining whether the second acoustic scene contains a sound in the first acoustic scene from a comparison of the representation of the second acoustic scene with the representation of the first acoustic scene.

20. The method according to claim 16 , further comprising conditioning a definition of the reference representatio

Assignees

• Apple Inc

Inventors

• Kim Hyung-Suk
• Klingler Daniel C
• Espi Marques Miquel
• Avendano Carlos M

Classifications

• G10L25/51

  for comparison or discrimination · CPC title

• G01H3/005

  Testing or calibrating of detectors covered by the subgroups of G01H3/00 (calibrating geophysical instruments, e.g. seismic receivers G01V13/00) · CPC title

• G06N20/00

  Machine learning · CPC title

• G08B7/06

  using electric transmission {, e.g. involving audible and visible signalling through the use of sound and light sources} · CPC title

• G08B21/182 Primary

  Level alarms, e.g. alarms responsive to variables exceeding a threshold · CPC title