By constant sound, I suppose you mean pure tone, i.e. a real valued sinusoidal.
Averaging raw bin values across DFT frames will not work properly unless the tones are in the same phase relative to the frame. This happens naturally if you have a tone that has a whole number cycles in each frame and the frames are back to back.
To make it clearer. Suppose there is a whole number of cycles in a frame, but you leave a half cycle gap to the next frame. The bin value in the second DFT will be completely out of phase (180 degrees, or $\pi$ radians) and the average will be zero as you would expect the magnitudes to be equal.
Even with back to back frames, if there aren't a whole number of cycles per frame the same type of interference effects will occur.
Therefore, you can only meaningfully average magnitudes across frames in most cases.
You are better off doing it all in one frame if the sound persists steadily across the interval. This will also separate the bin of interest from the bins of any other tones that may be in your sound. You don't need to calculate the entire DFT (aka FFT) if you are only interested in a few bins, they can be calculated individually.
If you are talking about multiple tones with various durations you might consider separate DFTs. A DFT works best on a steady tone across the entire frame.