What would happen if you took the STFT of a signal, discarded the phase information, and re-synthesized it? For example, just modulate oscillators tuned to each frequency bin by the amplitudes for those bins and add all of the results together for the output. Could human ears detect much of a difference? Would it make a difference depending on what kind of signal I pass through (say, a saw wave, a sound sample of a stringed instrument, and a voice)?

Your question is tricky because it is hard to define what "discarding the phase information" means without specifying a practical way of doing it; but then we encounter problems/artifacts which are specific to this particular method.

Since you mentioned STFT, let us assume phase information is suppressed by doing a STFT, keeping the magnitudes, setting to 0 or to a random number the angle, and doing the overlap-add resynthesis. You can actually hear something similar to that by using Paul's extreme sound stretch with a stretch factor of 1 (be sure to grab the python code to continue your experiments).

Two important observations:

• Pure tones get "phasey", "mushy", or appear modulated in amplitude. The reason is that if a signal contains a pure tone at frequency $f$, the relationship between the phase at $X(t, f)$ and $X(t + \Delta_t, f)$, where $X$ is the STFT and $\Delta_t$ the hop-size is certainly not arbitrary. Destroying this relationship will cause unwanted destructive interferences at the crossfade point of adjacent windows (giving the effect of a modulation at the analysis frame rate). Pure tones cannot exist without a coherent phase advance between successive frames.

• Within an analysis window, one effect of removing the phase information is that events localized in time will be smeared all over the window during resynthesis. The perceptual effect is very dependent on the window size. When large window sizes are used, percussive events lose their definition and sharpness and sound reverberated (imagine clapping your hands in an empty room, you don't have the impression of a single "hit", but of something spread in time).

All these are quite specific to the STFT implementation, and indeed highlight the challenges faced by the developers of STFT-based time-stretching algorithms - preserve the vertical phase relationships (to avoid transient smearing) and preserve the horizontal phase relationships (to keep pure tones sounding like pure tones).

You're probably asking this question because you have heard the ear is not sensitive to phase - it is true in the sense that it won't tell $sin (\frac{2 \pi t}{440})$ apart from $cos (\frac{2 \pi t}{440})$). But we are sensitive to phase coherence - the network of phase relationships the different elements in a time-frequency plot have between them. Any treatment breaking these relationships will be perceptible.

• Ah yes, I do see how percussive sounds would really be messed up, I hadn't thought of that. So I'm guessing all of the plosive sounds in speech would also lose a lot of their definition, as well as transitions between phones. – Void Star Aug 31 '12 at 20:02

At least 2 effects: transients will be smeared and any frequency content that is between FFT bin centers (not periodic in the STFT aperture length) will be distorted.

If you just happened to pick a note that was exactly periodic in the STFT window, then the latter effect might or might not not occur, depending on any slight inharmonicities in the overtone series, such as string instruments sometimes produce.