How to determine if certain notes exist within polyphonic music?

Question

Given a segment of audio, if you were to calculate the histogram of frequency amplitudes for all standard musical note frequencies present in the audio, how would you check to see if 2-3 specific musical notes exist in the audio?

This is a type of polyphonic detection, similar to this question. Except I'm not trying to comprehensively find all notes present in the audio. I already know what notes I'm looking for and am just trying to check to see if they're present.

My current (naive) approach is to:

1. Calculate the average amplitude across all frequencies to use as a threshold for noise filtering. Any frequencies with an amplitude below this I ignore as background noise.
2. For each note I'm searching for, I calculate the frequencies for the first 3 harmonics, lookup the amplitude for each of those frequencies, and if they're all above the average, then I assume that note is present.

I find this sort-of works, but isn't 100% reliable. The main problem I'm running into is that, given the type of musical instrument, the amplitudes of all the note's harmonics can be very inconsistent across the instrument's range, and this makes setting the noise threshold very error prone.

For example, on an acoustic guitar, playing the high E4 note, the fundamental frequency is very strong and larger than all the other harmonics. However, for the lower E2 note, the fundamental is so small, it's often excluded as background noise. And there's not always a consistent pattern within the bass strings. Some of the low bass string notes have a very strong fundamental as well.

How do I solve this? I know open-ended real-time polyphonic detection is a very difficult and unsolved problem, but are there any solutions for a constrained version where you're only checking for the existence of a few notes and their specific harmonics?

Answer 1

Since you don't want to collect audio samples the best I could suggest is to have some filtering of the compressed spectrum.

for instance

$$\underbrace{\int_{-\infty}^{\infty} \left( \sum_{k=1}^{\infty} u_k \cdot c(h(\omega/\omega_1) X\left( \frac{k\cdot \omega}{\omega_1}\right) \right) d\omega}_{\textrm{Harmonics}} + \underbrace{\int_{-\infty}^{\infty} \left( \sum_{k=2}^{\infty} v_k \cdot c(h(\omega/\omega_1) X\left( \frac{\omega}{k\cdot \omega_1}\right) \right) d\omega}_{\textrm{Subharmonics}} $$

Where

• $h(w)$ is a function that will take the spectral leakage in to account when scoring each harmonic/subharmonic.
• $u_{k}$ is a weight to the harmonic $k$
• $v_{k}$ is a weight to the hamonic $v$
• $c(I)$ is a function that translate amplitude to importance, maybe you want to use a logarithm function since human perception of sound intensity is rouhgly logarithmic. Maybe simply a square root to avoid detecting a harmonic of the fundamental, for instance suppose that some timber has a very strong third and fourth harmonics and you play a C4 and a G3, if you just search for use intensity directly you could detect a G5. With this compression function you will give more importance to the detection of multiple harmonics instead of looking to the intensity of a particular harmonic.

The sums and the integral are declared to be infinite, but since $u_k$ and $v_k$ tend to zero as $k$ increases, you can set a limit for the summations. And since the signal has a limited spectrum you can set a bounds to the integration.

Then you guess all of these parameters without collecting samples, and if you are lucky you will have a good result. If you are doing this based in your prior experience you are implicitly using collected data as well.

Answer 2

For those 2 or 3 specific notes that you're trying to detect, you could have 2 or 3 specific comb filters, each tuned to the fundamental frequency of each of the 2 or 3 specific notes.

If that particular comb filter lights up, it's a strong possibility that you've detected the note that the comb filter is tuned to.