1
$\begingroup$

I should start this off by saying I’m a hobbyist and by no means a student. I’ve been reading “The Audio Programming Book”, and attempting to implement STFT on the stm32 based Daisy platform.

In order to learn from a working sample, I’ve been using mutable instruments’ Shy_fft for fft processing. So far, I have normal fft > Ifft functioning and am beginning to take steps towards STFT.

I cannot seem to get the stft functionality working, and am struggling to even understand if I’m processing / understanding the fundamentals.

Here is the second implementation I tried. The first version was overwritten by the update

The first version seemed to “work” but there was an issue where the output was “choppy”, seeming to be dropping frames. The second version didn’t seem to output any audio.

EDIT: I should mention that the gist wasn’t updated with a functioning audio output. I’ll try and update it later and remove this edit

Any tips/comments on where I’m going wrong/how I should be doing things are greatly appreciated.

EDIT 2: See updated NaiveSTFT class here

$\endgroup$
23
  • $\begingroup$ In the first link, in the process function, you check separately for the fft_buffer_index and ifft_buffer_index, aren't they supposed to be the same through initialisation? Not that this will solve your problem but if you don't provide some information on how you use the class I am not sure it will be very easy to find where the problem is. Could you provide a minimal working example? $\endgroup$
    – ZaellixA
    Sep 17, 2023 at 11:22
  • $\begingroup$ I believe that in the second link, you provided, line 102, the fft_.Direct() function should be outside the inner for-loop, as, if I understood the implementation correctly, it is supposed to act on the whole buffer (which is FFT_SIZE long and offset based on the "hop size" and frame - which seems correct -), which means that it should be called when the windowing process is finished, which in turn is outside the inner for-loop of line 91. I haven't run the code so I am not sure this may be an issue but based on my understanding it seems like a problem to me. $\endgroup$
    – ZaellixA
    Sep 17, 2023 at 11:26
  • 1
    $\begingroup$ You may be right! I didn’t think about processing outside of the loop, due to counting through the frames. I’ll give it a go later. Also, I’ll provide the class in context of its use as well. I should have thought to do that! $\endgroup$ Sep 17, 2023 at 11:31
  • $\begingroup$ This is the right track @ZaellixA! So now I'm getting audio out again! The issue now is it seems there's some underlying signal that sounds kind of like a low oscillation on the output, as well as the "choppy" audio quality. Here's the updated class and here's the actual implementation $\endgroup$ Sep 17, 2023 at 16:02
  • $\begingroup$ There are various things I'm missing here, but I'll start with those I think are the most important. You are multiplying with the window function the output of the IFFT, are you sure this is needed (implementation for-loop starting on line 124)? I believe you should apply an overlap-save/overlap-add technique here instead of windowing the data. This will introduce consecutive Hann windows in the time domain which will most probably sound like very short successive sounds (since you seem to be using a very short frame size). $\endgroup$
    – ZaellixA
    Sep 17, 2023 at 19:22

0

Browse other questions tagged or ask your own question.