Let’s assume that the audio input will be:

  • Guitar audio
  • Monophonic audio

Is the STFT overkill? Is it enough to produce a time-frequency plot and from there find the notes played at each instance? What problems could arise?

  • $\begingroup$ plucked strings have inharmonicity, so the fundamental frequency you'd get from an FFT/STFT is not necessarily the pitch we hear. (weighted?) autocorrelation peak should be more similar to what we hear, since our ear nerves do autocorrelation supposedly. $\endgroup$ – endolith May 8 '17 at 16:15

No. An STFT is not only not overkill, it is not enough to find guitar "notes" reliably. An STFT measures energy at selected spectral basis frequencies. But a note usually refers to psychoacoustic perceived pitch, or pitch frequency, which is very often not the same as the spectral frequency peak, especially for guitar notes. Guitars can produce more sound power at various (potentially slightly inharmonic) harmonic frequencies than at the perceived pitch frequency. Also, the STFT basis vector frequencies may not accurately correspond to any of the musical frequencies of interest (being between them instead).

Try looking up pitch detection/estimation algorithms instead of using an STFT or bare FFT magnitude.

  • $\begingroup$ Thanks. Do you think that methods such as autocorrelation will be more reliable? Let's say I am given a spectrogram of an guitar, monophonic, audio file. How should I proceed to find the notes played? $\endgroup$ – pavlos163 May 7 '17 at 21:55

i actually get to disagree with hotpaw about this. in a sense, i believe that STFT is both too much and not enough for pitch detection of monophonic quasiperiodic tones.

pitch detection is an operation that is lossy regarding your audio data. that is many many different tones that sound wildly different will have the same pitch, a good pitch detector will return that pitch and you can't take only the pitch information and reconstruct the tone (with the same loudness and timbre). the STFT retains all the information (and some redundancy because of overlapped windows) and to extract a single parameter (the period length at a precision of fraction of a sample)is a bit of overkill.

and that overkill isn't free in cost. if your guitar pitch detector is intended to be running real time, i don't think you can afford the overhead in delay, and your processor (whether DSP or an ARM or something) is going to be very busy with an FFT happening every few milliseconds. in terms of delay, you have to first fill the buffer with samples before you pass it to the FFT. even if your computer was infinitely fast, whatever period or pitch result returned will apply to the buffer that was most recently completely filled. using a say 2048-sample FFT, that means 50 ms minimum (for $f_\text{s}$=48 kHz).

lastly (about the lackings of STFT pitch detection), you have to put in some squirrelly AI algorithm to deal with missing harmonics or a missing fundamental. what you want out is pitch (as perceived by a human listener), not necessarily what the individual frequency components are. if you're not careful, you could have a note that begins with one pitch and is sustained with feedback or whatever and it morphs into the note an octave above. what's happening is that the fundamental and other odd harmonics are dropping to zero while the even harmonics are not dropping as fast. how do you want your pitch detector to behave in this case? do you want it jumping up an octave when you're just sustaining a note and not playing a new note?

take a look at these pages regarding Axon 1, 2, 3, and this paperc=icmc;idno=bbp2372.1999.332). they used to have a page with a drawing of how they matched slopes of the wave right after the picking of the string and the wave that was one period since. i think they had a delay of 13 ms, which is pretty good since the low E on a guitar is 12 ms.

i wish i could find the patent or remember the name of the guy who developed the alg. i will have to sift through some email to recall.

if you want to get your pitch detect result as quickly as possible, you'll do either AMDF or ASDF or some form of autocorrelation, where you will be comparing the samples in the buffer that are most recent to samples in the past and the difference between the two segments would be the lag value of the correlation. these methods make no assumption that the fundamental (nor any other single harmonic) is present.

  • $\begingroup$ Although neither a window of time domain sample or its STFT result lose information, neither directly results in pitch without further processing, autocorrelation/AMDF/etc. in the time domain, or cepstral/HPS/etc. in the STFT or frequency domain. Both domains may require even further post processing to make threshold and/or octave decisions. $\endgroup$ – hotpaw2 May 8 '17 at 13:46
  • $\begingroup$ I am now using freq_from_fft but I get octave errors. What kind of post-processing could I implement to fix that? $\endgroup$ – pavlos163 May 9 '17 at 19:07
  • $\begingroup$ i wouldn't use STFT at all. i would use some form of autocorrelation. specifically what is depicted here and here. $\endgroup$ – robert bristow-johnson May 9 '17 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.