In order to remove the noise from the signal, you need to subtract the spectral content of the noise from that of the signal ( filtered_signal = unfiltered_signal - noise ). This technique is called spectral subtraction. This will only work well if the noise is rather uniform across the entire length of the signal.
The steps required to do this are
Identify the noise region(s) in your signal and do an FFT on it. A single noise region is enough provided that it's long enough for your FFT.
Do the FFT on the entire signal. This may be done in multiple segments (frames).
In the frequency domain, subtract the result of step 1 from step 2. If you're using multiple segments in step 2, then simply subtract the result from step 1 from every segment. This will effectively filter your signal.
Do an inverse fft, which will bring your signal back into the time domain.
Note that I said that this technique will only work if the noise is uniform and by uniform I mean that its spectral content is rather constant. But you can also have regions in your audio where the noise is not present at all so obviously trying to remove the noise from these noiseless regions should be avoided. If your noise removal process is manual (at least in part), it would be possible (albeit tedious) to label such regions and then have the algorithm skip them.