To record sound and transform it to digital form, audio signals are recorded and its pressure levels are sampled and quantised.
Why does quantisation noise sound like white noise?
It often doesn't sound like white noise. The quantization noise of a sine wave is fairly harmonic with a clearly detectable pitch. The quantization of a pulse train sounds like a pulse train.
If the quantization has enough bits, and if the signal is reasonably stationary in time and reasonably dense in the frequency domain, the quantization noise will be more or less white. For music quantized at 16 bits, that's often the case. However the quantization noise will always be correlated with the original signal which can produce undesirable artifacts. Hence commercial audio production uses dithering and noise shaping and decrease the audibility of quantization.
(following Olli's suggestion, posting my short answer): As long as the waveform transitions through multiple quantization levels per sample, and if the waveform is not correlated to the sampling clock, then the remainder to the closest sample boundary (which is the quantization error) is independent, uncorrelated from the remainder in the previous sample, which by definition is white noise. (Although uniform white noise in this case if there is an equally likely chance that the actual signal can be anywhere between two quantization boundaries). Any correlation between the signal and the sampling clock will result in pattern repetition of the error signal, and pattern repetition results in discrete frequency correlations that we call "spurs".
Please see the good work Olli Niemitalo has detailed on quantization noise here:
It does not! The rounding errors introduced by quantisation are dependent on the audio, and when the input waveform has a repeating pattern, the quantisation errors will have a repeating pattern as well.
This is why we add dither: extra random noise before rounding. If our random noise follows the right distribution, we can ensure that the rounding error for each sample is (1) not correlated with errors at other samples, and (2) has constant mean and variance. When the rounding errors have these properties we perceive it as white noise.
Here is a page with some audio demos so you can hear non-white-noise quantisation errors, and how dither can improve the sound: http://www.earlevel.com/main/2010/11/07/the-sound-of-dither/