Although the specifics of your data will be the ultimate factor in how this sort of normalization will impact what you're trying to measure, I think it's useful to look at extreme cases to get a sense of what might go wrong.
For example, suppose the time samples of your STFT just happened to come out one-hot, with 1s in either the highest-frequency or lowest-frequency bins and 0s everywhere else :
STFT = [[1, 0, 0, 0, 0],
[0, 0, 0, 0, 1],
[1, 0, 0, 0, 0],
If you subtracted out the mean values from each time sample, then clearly you'd get some "bleeding" of the power information across frequencies. In particular, the middle-frequency bin would get some nonzero value, even though in your unnormalized STFT values, that bin never has any power.
What about just dividing each time sample by its standard deviation, like you're asking about ? In this case you'll just be changing the value of the nonzero frequency bin. This would certainly impact the mean value across time of each frequency bin, since all nonzero values in your spectrogram would now be 1+$\epsilon$ rather than just 1.
This is just one possible thought experiment that you could do to explore these effects. I'd say in general that the statistics of the frequency bins across time would definitely be affected by normalizing each time frame independently, but again that the specifics will depend heavily on the data that you're analyzing.