I'm trying to perform automatic pitch tracking for piano. It's easy to detect the note when there's only a single note pressed. There are 88 keys on the piano and I set the result to one of them. There may be misclassifications due to octave shift. For example, instead of 40, I find 52. They are both C, but 52 is one octave up. I can remove most of these octave shifts by performing median filtering on the results.

But it is confusing when I move to polyphonic pitch tracking. When there are more than a single note pressed together, there can be varying number of pitches detected at time t.

For example, let labels(f,t) be 1 if note f is detected at time t and let the binary matrix labels it be as follows

labels = 0 0 0 0 1 ...
         1 1 0 1 0 ...
         0 1 0 0 1 ...
         0 0 1 0 1 ...
         0 0 0 0 1 ...

Let the detected notes be as follows correspondingly:

labelsAsNotes = 2; 2 3; 14; 2; 1 3 14 88; ...

How can I do median filtering in this case? Do you have any other recommendations to remove octave shifts?


1 Answer 1


I don't think median filtering is a good approach for this problem. It is definitely used as a smoothing mechanism in single pitch tracking, but I don't think that's necessarily the best way to go. For example, when tracking single pitches, a dynamic programming algorithm can be really powerful. At each frame of audio you have pitch candidates with some strength to them. Dynamic programming lets you find an optimal path between those candidates such that you penalize big jumps in pitch (like a whole octave) while promoting pitch candidates with high weights.

As far as I'm aware, though, this doesn't scale well to multiple pitches. I have seen authors use this algorithm to track just two pitches in the case of two concurrent speakers.

This is a really hard problem to solve in general, and I don't think just filtering that binary matrix will be sufficient -- I think you want some sort of weights associated with each pitch candidate, and to make decisions with those weights. You can also take time into account -- if you detect two notes that are an octave (or other integer multiple) apart, and follow the same relative pitch trajectory and onset at exactly the same time, they are probably the same note. You could also try using the weights to inhibit octave errors -- for example, if you detect a note, but there's significant energy at half its frequency, you might be looking at an octave error.

  • $\begingroup$ +1 Thanks for the nice answer. I have got weights related to the likelihood of each piano button. I will try to perform median filtering on the weights independently and try to see how it works. Dynamic programming seems to be a good way to go, but as you said it is not straightforward in the multi-label case. $\endgroup$
    – petrichor
    Commented Jan 18, 2012 at 13:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.