What are you planning to do with the STFT?
The time-domain mean of the signal is the DC component and this should usually be close to zero for audio. Any DC offsets or low frequencies below the threshold of hearing (eg. around 20Hz, give or take) are generally considered defects and usually removed by filtering during production if present to a significant degree, since they usually serve no purpose other than wasting some dynamic range. Microphones don't record it and speakers don't reproduce it. In fact most commercial music is typically high-pass filtered closer to 40-50Hz as few consumer systems can reproduce meaningfully below that.
Windowing of the STFT will usually still give you a slightly non-zero value in the DC bin, but this is usually quite small, at least if we the window size is reasonable compared to the wavelengths of the frequencies present. For algorithms such as fast convolution it's important to preserve and processes this value as-is such that frames align correctly. This is also true for any spectral processing that works with windows that are shorter or similar to the wavelengths involved (eg. less than 100ms or so), yet seeks to reconstruct the original signal.
For typical audio analysis duties there is usually no need to explicitly remove it, since it's presence can just as well be ignored. If you are working with very short windows, then spectral leakage might be an issue, but often in audio one is stuck working with fairly large windows (to get some sort of resolution at the lower end of hearing) and/or window functions with good dynamic range (to deal with the more significant spectral leakage at frequencies actually present), so spectral leakage from DC is usually essentially irrelevant (or alternative the apparent spectral leakage around DC is more likely to come from side-bands of the actual desired low frequency content).
So in general I would say "no" with the caveat that the possible applications of STFT in audio are quite varied and you likely need to evaluate each specific application separately, especially when working with short windows, where as with longer windows it usually doesn't make a whole lot of difference, because the expectation is that the mean is zero.
Clarification on common formats based on the comment:
Logically the audio signals are usually treated as normalized values in range [-1,+1] where $\pm1$ are the clipping thresholds (ie. 0dB "full scale"). In floating point formats the values are usually stored as such, while in integer formats the values are treated as fixed point such that the minimum value is treated as -1 and the maximum value as one quantization step below +1 (though especially in software the maximum positive value is sometimes treated as +1 and the minimum value as one quantization step below -1, but this is generally not very significant as long as one is consistent).
For bit-depths of 16-bits or higher, the values are usually (eg. WAV) stored as signed, but 8-bit samples are sometimes (eg. WAV again) stored as unsigned. When stored as unsigned, the values still logically represent bipolar signals such that the designated mid-point (eg. usually 128 for 8-bit unsigned) represents the logical value of zero; in this case the signed fixed point value is simply biased into the unsigned range and should be reverse-biased back into a signed value before processing.
Hope this helps.
