Testing if a signal contains a specific previously recorded signal

Question

I am working on a project that converts radio data and transcribes it into text looking for specific types of radio chatter. For most of the time, the radio signal is dead air. Periodically, there will be a callout for a specific type of municipal event (fire, medical, police). Each callout has unique series of sounds that proceeds the call that identifies the type. To the ear, they sound the same. But the signals aren't exactly the same each time, due to noise/distortion, volume.

What I'm trying to do is identify a way to automatically categorize each callout by it's municipal type based on the preceding sound. The current technique I have utilized is first taking the raw signal data in 15 minute clips of mp3 data. I cut it into clips by removing the silence. I then take the first 4s of the clip, convert it into a spectrogram using imagemagick, and then use imagemagick's compare -metric phash mechanic to compare it to a previously saved spectrogram that was recorded earlier for a known valid signal. The solution works OK, but not great. It causes false positives that to the ear are not even close. If I reduce the hamming distance that I allow, it causes me to miscategorize valid callouts. If I increase the hamming distance, I get false positives. The other issue is that it takes a lot of processing (raw -> remove silence -> for each clip -> get 4s clip -> spectrogram -> compare -> if match -> save clip). Ultimately it seems like the methodology is flawed... there has to be a better way.

Here is an example file https://drive.google.com/file/d/1GcU6zQDr2G39a5Szpr-A6rPBYjv9Zrch/view?usp=sharing . It is 15 minutes of signal data. If you skip to 1:52 you will hear the sound for a medical call out, it's a single pitch escalating in volume, same tone every time. Another medical call happens at 2:07. If you skip to 3:44 you'll hear the preceding sound for a fire callout. It's a weird sound that almost sounds like an old school modem, but it's the same for each fire. There is another fire at 5:35. The wave forms for the fire calls outs are similar but not exactly the same. Prior to this project I knew absolutely zero about signal processing, and I still know just barely above zero. My gut instinct tells me there must be someway to smooth the waveforms and then do some sort of mechanic where it compares the shape of the waveform, but not the exact height (so if one callout is just barely louder or quieter they match). Something like a regex but for signal data, but I have no idea if that even makes any sense or if there isn't a better way. When I've looked online a lot of it has been stuff like Audio finger printing, but that hasn't made sense to me because I'm not trying to identify an entire file, but rather see if my raw file contains this audio file. It seems similar to the algorithm used in Shazam, but I can't find the actual algorithm in that paper, it just seems to talk about generating the constellation, but not the specifics of how to actually generate the constellation.

My concern boils down to two main problems. First, what's the right algorithm for this, and second how do I actually execute it. I have experience/access to python, node and golang. If I knew the right algorithm I think I could figure out the rest, but if there is a well-known library that already tackles this, even better.

Edit:

I've made some progress based on the answer from Max. But I'm still stuck at an impasse.

testFile, sr = librosa.load("/app/src/1601317821107.mp3")
matchFile, sr = librosa.load("/app/src/medicSound3.wav")
corr = scipy.signal.correlate(testFile, matchFile, mode="full", method="fft")

If I plot corr I get the following graph.

That clearly shows 4 matches in the given file, which is exactly correct. If I run it against a file with no actual matches, then the graph never has any peaks with higher than 10. So the next challenge is how do I locate the start point of each of those peaks. I've tried scipy.signal.find_peaks but that gives me lots of peaks, not just the 4 I seek. I've tried fiddling with the prominence, width, height params but none of have really found what I'm looking for. In this case, I'm looking for the start of each of the major matches. If I knew the start index I believe I can simply divide that by sample rate (22050) to get the position in seconds. The other problem is that the corr array is the length of both the testFile and matchFile arrays combined. Which causes my indexes on the corr array to not exactly match their location in the testFile array.

Thoughts?

Answer 1

This does sound like a job for cross correlation. It's quite robust and some tweaking with the threshold value should give you the result you are looking for. It's calculated in the time domain, so no spectrum or spectrogram fiddling neccessary. Basically, you just "slide" your sample over the recorded audio and do a convolution of the overlapping parts. It will have several peaks, when sine tones are in play, but usually, the highest peak will give you the exact position of the signal you are looking for. If you only need to know if it occurs at all, you could make it even more robust by some sort of thresholding not samplewise, but over the sum of the whole correlation analysis.

Edit: Of course, there is a well documented, widely used python library for this: numpy

Edit 2: I would tackle the non-matching length by passing "same" to the function instead of "full". This will yield a signal of the same length as the test signal. Next, I would abs the signal and then run a scipy.signal.medfilt over it, with kernel_size about the length of the match signal. This should give you a clear peak for each occurrence, which you can then identify as you tried before.