Which (Fourier transform?) processing method to use for time variant audio processing? (newbie)

Edit: I've since been able to understand the how the relvant parts of Fourier transform work and how they relate to my problem / what I want to do. And have come to the conclusion that Fourier transform is not the best suited to my problem and that I have an alternative way to do what I want that is much simpler to control / behave in the way I want in practice.

Since I'm new here I don't know what's the right thing to do with my question on this site. Just in case anybody was following this / was preparing an answer, I'm leaving this question up for one more day with this edit and after that I'll delete the question.

I've devloped a basic algorithm for processing audio. I have some programming experience but none with Fourier transforms (but will dedicate all of 2020 to learn this full time if need be). My question relates to how best to program my algorithm in practice.

What I need to do is the following:

• Analyze the frequency content at exact locations of a 44.1kHz PCM audio stream / file.
• Detect and modify the amplitude of specific frequencies at specific time locations (I do not need to modify phase or have knowledge of phase to do this).
• Then turn this modified version back into audio without audible artifacts from processing errors/limitations.

As I said, I currenty do not know much about Fourier transform. So far my understanding is that an FFT (with windowing) will allow me to analyze the frequency spectrum of a block, detect and modify the amplitude of specific frequencies. But not modify the amplitude of specific frequencies at a specific time in that block and not the same frequency at a different time in that same block? For instance if for 44.1kHz sample rate and block size 8192 as far as I understand it normal FFT will not allow me to detect for instance two seperate 10kHz transients in that block as seperate and modify only one of them?

If my above understanding is correct. Can anybody suggest proper (Fourier transform like?) processing method(s) to use / investigate to achieve my goal?

I've throught of a rough way to do what I want with FFT/IFFT by dividing the frequency spectrum (5Hz to Nyquist for instance) into seperate octave bands all processed seperately, for instance first band 11.025kHz to 22.05kHz under 44.1kHz sample rate and block size 4 for instance not sure of the right block size just an estimate (so I get the transients in one block and still sufficient frequency resolution for that frequency band), then 5.5125kHz to 11.025kHz band under 22.05kHz sample rate (downsampled) also with block size 4 as an estimate, etc with a total of 12 bands down to about 5Hz for instance. But this will be a lot of work and will take me a lot of time to develop and surely someone must have done something similar and smarter a long time ago to do the type of processing I wish to do :) If anybody can point me in the right direction (and perhaps if I got some assumptions wrong correct me) I will be very thankful!

• Some questions before answering: 1. Does it have to be real time? 2. You mention in the beginning that you have developed a basic algorithm. What is the basic algorithm you have developed? Can you describe it? 3. Is it mandatory to be FFT-based? 4. Finally, what exactly do you want to do? I mean, what is the purpose of modifying the amplitudes? What is the target application (let's say)?
– GKH
Jan 4, 2020 at 22:31
• Hello @GKH 1. In the end I do want it to be real time (though a large latency is acceptable), it may be processor intensive. If it takes a whole computer or a seperate powerful DSP unit to run it that's ok. 2. I cannot describe the algorithm in detail as it is something new. If I describe it there's a good chance someone else will write it before me. I'm certain though that the algorithm itself is correct. 3. No, it does not have to be FFT based as long as I can modify the amplitude of specific frequencies at specific points in time. Jan 5, 2020 at 14:14
• @GKH 4. Part of my algorithm needs to 'substract' certain audio from one file from another. If a certain frequency is in audio file 1 and at the same time in audio file 2 I need to reduce the volume of that frequency in audio file 1 by the amount of that frequency being present in audio file 2. This is regardless of phase of those frequencies, and if the specific frequencies are (individually) louder in audio file 2 than in audio file 1 then they should be silent (not reverse phase or anything). Jan 5, 2020 at 14:22
• @GHK Think for instance two independent channels of pink noise. If channel one and channel two have an equal frequency at the same time I want to be able to reduce the amplitude of this in channel 1 by the amount it is present in channel 2. I think to be able to do this there's a balance between time resolution and frequency resolution (and possibly amplitude detection accuracy)? I think I will need it to have fairly high time resolution and only moderate frequency resolution? (maybe even as wide as a quarter octave would do?) I do really need the artifacts of the processing to be inaudible. Jan 5, 2020 at 14:27
• @GHK Nevermind my comment "(maybe even as wide as a quarter octave would do?)" that doesn't make sense. I have no idea yet what frequency resolution I would need or how FFT handles this. Thinking about it, for short transients the frequency resolution wouldn't need to be small but for long sounds the resolution would need to be small as the ear/brain can then detect them as individual frequencies? Jan 5, 2020 at 14:42