Detection and classification of siren signals and localization of siren signal sources

Patent metadata

Field	Value
Publication number	US-11295757-B2
Application number	US-202016752595-A
Country	US
Kind code	B2
Filing date	Jan 24, 2020
Priority date	Jan 24, 2020
Publication date	Apr 5, 2022
Grant date	Apr 5, 2022

Abstract

In an embodiment, a method comprises: capturing, by one or more microphone arrays of a vehicle, sound signals in an environment; extracting frequency spectrum features from the sound signals; predicting, using an acoustic scene classifier and the frequency spectrum features, one or more siren signal classifications; converting the one or more siren signal classifications into one or more siren signal event detections; computing time delay of arrival estimates for the one or more detected siren signals; estimating one or more bearing angles to one or more sources of the one or more detected siren signals using the time delay of arrival estimates and a known geometry of the microphone array; and tracking, using a Bayesian filter, the one or more bearing angles. If a siren is detected, actions are performed by the vehicle depending on the location of the emergency vehicle and whether the emergency vehicle is active or inactive.

First claim

What is claimed is:

1. A method comprising:
capturing, by one or more microphone arrays of a vehicle, sound signals in an environment;
extracting, using one or more processors, frequency spectrum features from the sound signals;
predicting, using an acoustic scene classifier and the frequency spectrum features, one or more siren signal classifications;
converting, using the one or more processors, the one or more siren signal classifications into one or more siren signal event detections;
computing time delay of arrival estimates for the one or more siren signal event detections;
estimating, using the one or more processors, one or more bearing angles to one or more sources of the one or more siren signal event detections using the time delay of arrival estimates and a known geometry of the one or more microphone arrays; and
tracking, using a Bayesian filter, the one or more bearing angles,
wherein predicting, using an acoustic scene classifier and the frequency spectrum features, one or more siren signal classifications, further comprises continuously predicting labels indicating the presence or absence of the one or more siren signals and their respective start and end times.

2. The method of claim 1 , wherein the time delay of arrival estimates are computed using a maximum likelihood criterion obtained by implementing a generalized cross correlation method.

3. The method of claim 1 , further comprising:
estimating one or more ranges of the one or more siren signal sources by applying triangularization to the one or more bearing angles.

4. The method of claim 1 , wherein transforming sound signals into frequency spectrum features includes generating one of a spectrogram, mel-spectrogram or mel-frequency cepstral coefficients (MFCC).

5. The method of claim 1 , wherein the acoustic scene classifier is implemented at least in part using a convolutional neural network (CNN).

6. The method of claim 1 , wherein the Bayesian filter is one of a Kalman filter, extended Kalman filter (EKF), unscented Kalman filter or particle filter.

7. The method of claim 1 , wherein the one or more bearing angles are estimated by using a spatio-temporal difference of the one or more siren signal event detections at each microphone pair in the microphone array.

8. The method of claim 1 , wherein the acoustic scene classifier predicts labels that indicate the presence of different types of siren signals.

9. The method of claim 8 , wherein the different types of siren signals include at least wailing, yelp, hi-lo, rumbler, and mechanical wail siren signals.

10. The method of claim 1 , wherein a plurality of the one or more bearing angles are used to triangulate the location of the sound source.

11. The method of claim 10 , wherein the vehicle is an autonomous vehicle, and wherein the sound source is associated with an emergency vehicle and the method further comprises:
causing, using the one or more processors and the location of the sound source, to operate the autonomous vehicle in accordance with one or more rules associated with emergency vehicles.

12. The method of claim 11 , further comprising: determining whether the emergency vehicle is active or inactive, and if active, operating the autonomous vehicle in accordance with a first set of rules associated with active emergency vehicles, or if inactive, operating the autonomous vehicle in accordance with a second set of rules associated with an inactive emergency vehicles.

13. The method of claim 12 , wherein if the emergency vehicle is active and nearby and the autonomous vehicle has crossed a stop line at an intersection, causing the autonomous vehicle to initiate a comfort stop, or if the emergency vehicle is active and far away and the autonomous vehicle has crossed the stop line, causing the autonomous vehicle to traverse across the intersection and then initiate a comfort stop.

14. The method of claim 12 , wherein the emergency vehicle is active and the autonomous vehicle is within a left lane, and a right lane is open and available, causing the autonomous vehicle to merge into the right lane.

15. The method of claim 12 , wherein the emergency vehicle is active and the autonomous vehicle is within the rightmost lane, causing the autonomous vehicle to bias to the right, if possible but not cross the lane boundary, and then initiate a comfort stop and remains stopped until the following conditions are met: 1) the emergency vehicle is traveling away from the autonomous vehicle with a range rate that is greater than a specified speed for greater than a specified time; and 2) the emergency vehicle range is greater than a specified distance, or the emergency vehicle is no longer detected for greater than a specified time, and if the conditions are met causing the autonomous vehicle to resume its route towards a goal point.

16. The method of claim 12 , wherein the emergency vehicle is inactive, the method further comprising:
determining whether the autonomous vehicle is on a same road as the emergency vehicle;
determining whether the autonomous vehicle is on the same side of the road or an opposite side of the road as the emergency vehicle;
if on the same road as the emergency vehicle,
determining whether the autonomous vehicle is in in front or of behind the emergency vehicle; and

causing the autonomous vehicle to travel a trajectory to avoid collision with the emergency vehicle based on whether the autonomous vehicle is on the same road or a different road, the same side of road or the opposite side, and if on the same side of the road whether the autonomous vehicle is in front of or behind the emergency vehicle.

17. The method of claim 12 , wherein the emergency vehicle is inactive, further comprising:
determining that the autonomous vehicle is on a road with single lane, and the emergency vehicle is located fully on a shoulder area and fully or partly within a lane in which the autonomous vehicle is traveling; and
causing the autonomous vehicle to initiate a comfort stop is initiated; and
causing the autonomous vehicle to remain stopped until the emergency vehicle is located fully on the shoulder area, and then to proceed with a maximum speed limit or switch to active and travel away from the emergency vehicle.

18. A system comprising:
one or more microphone arrays;
one or more processors;
memory storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations comprising:
capturing, by the one or more microphone arrays, sound signals in an environment;
extracting frequency spectrum features from the sound signals;
predicting, using an acoustic scene classifier and the frequency spectrum features, one or more siren signal classifications;
converting the one or more siren signal classifications into one or more siren signal event detections;
computing time delay of arrival estimates for the one or more siren signal event detections;
estimating, using the one or more processors, one or more bearing angles to one or more sources of the one or more siren signal event detections using the time delay of arrival estimates and a known geometry of the one or more microphone arrays; and
tracking, using a Bayesian filter, the one or more bearing angles;

wherein predicting, using an acoustic scene classifier and the frequency spectrum features, one or more siren signal classifications, further comprises continuously predicting labels indicating the presence or absence of the one or more siren signals and their respective start and end times.

Assignees

• Motional Ad Llc

Inventors

• Buddhadev Jugal
• Venkateswaran Rajesh K
• Sharma Alok
• Xu Yunpeng
• Lee Michael
• Schmitt Paul
• Andromalos Matthew

Classifications

• G06F2218/12

  Classification; Matching · CPC title

• G06F2218/08

  Feature extraction · CPC title

• G06F2218/02

  Preprocessing · CPC title

• G06N3/045

  Combinations of networks · CPC title

• H04R2499/13

  Acoustic transducers and sound field adaptation in vehicles · CPC title