What Are The Semantics Of Wav-File Sample Values?

Question

The title is the question. To make this concrete, assume I have a standard 16-bit single channel wav file, with (although it doesn't matter) a 16000 Hz sampling rate.

Each sample magnitude will be a 16-bit value.

Is there a specified semantics or interpretation of these values, not relative to any physical value, but relative to the numerical systems of modern computers? I.e., are those numbers to be interpreted as 16-bit twos-complement, or scaled into single precision floats between -1 and 1, etc?

At the highest level, I recognize this is a little pointless: They're arbitrary scale.

But at a practical level, in the python world, librosa does one thing, scipy does another, and converting those different things into a spectrogram or analyzing with a vocoder can yield different results.

For skeptics, invite you to try the following:

import scipy
import librosa.display

np_eps        = np.finfo('f').eps
np_max        = np.finfo('f').max

x, sr = librosa.load(file, sr=16000)

plt.subplot(2, 2, 1)
D = librosa.amplitude_to_db(np.abs(librosa.stft(x)), ref=1.0)
librosa.display.specshow(D, y_axis='linear')
plt.colorbar(format='%+2.0f dB')

plt.subplot(2, 2, 2)
plt.plot(x)

sr, y = scipy.io.wavfile.read(file)
y = y.astype(float32)
plt.subplot(2, 2, 3)
D = librosa.amplitude_to_db(np.abs(librosa.stft(y)), ref=1.0)
librosa.display.specshow(D, y_axis='linear')
plt.colorbar(format='%+2.0f dB')

plt.subplot(2, 2, 4)
plt.plot(y)

Do not neglect the extra argument to librosa, forcing the sampling rate; in my case, the file I am using is from the Arctic corpus and is sampled at 16 KHz. Do not neglect the colorbars, which will show you the very obvious shift of 90 dB from one to the other. This should not be surprising, since 20 * log10 (32768) is approximately 90.3.

I assure you, gentle readers, this will have consequences further down the pipeline of speech vocoders, or neural networks (where normalization techniques might want you to reference to numpy epsilon to avoid negative numbers) or both.

Answer 1

I don't know much about this semantics? of WAV files but their numerical format is the following. (assuming mono)

• Given a recording with 8-bit per sample precision, then those samples are unsigned integers taking values between $0$ and $255$. Due to being unsigned, to represent negative values, there is a bias of $128$, and the sample values are actually interpreted to be between $-128$ and $127$. In this case a value of $128$ represents silence, maximum value $255$ corresponds to an input full scale positive audio waveform and $0$ represents neagitve full scale input waveform.

• For $16$-bit WAV files, you have $16$-bit signed integers that range from $-32768$ to $32767$ and $0$ represents silence. Again, $32767$ represents full scale audio input without scaling.

Note that here I refer to a microphone recording through an ADC system which converts analog audio input in the standard range of about $1$ V rms into WAV samples. Please check this 1-V rms for being something higher, about 1.5 Volts or else depending on particular implementation or some specific audio standard. In any case, the analog circuitry is responsible for transmitting a full scale audio to the ADC whan a 1V (or 1.5V) rms wave is present at the input and then the conversion is performed according to the above setting when WAV recording is performed.

Answer 2

The semantics for a sampled audio signal is very simple. Each sample represents an amplitude, each sample is done at a specific time.

If you create a signal using a microphone, the amplitude is related to the pressure as measured by the microphone diaphragm. The sampling process will introduce time in the equation.

In the question, there are two sets of diagrams. The right hand diagrams shows the signal amplitude although the individual samples are difficult to see there (you may magnify). You might see it as the deflection a loudspeaker cone would do.

The left hand diagrams is the result of a Fourier transform, where the signal is presented as both a frequency and a "loudness". You could see this as a representation closer to what our mind perceives when listening.

The 16 bit values in the wav file are amplitude values, encoded as signed 16 bit numbers. The amplitude can go from -32768 to +32767. The value can often better be interpreted as (almost) -1.0 to +1.0, but this is an interpretation.

In audio we may interpret a maximum signal, swinging from -1 to +1 as having a loudness of 0dB. Smaller signals would have negative values, say -20dB.

Audio signals tend to have a mean amplitude of zero. This comes from the high pass filter of the audio circuits in front of the AD-converter. Most often we do not want to include signals below something like 20 Hertz. A mean value outside of zero would indicate a DC signal, frequency 0Hz.

The number of bits in the samples, here 16, is related to the maximum Signal to Noise ratio, known as S/N, that the signal theoretically can have. Very simplified, each bit allows about 6dB of possible S/N. A maximum swing signal, going from -1 to +1 would then approach a S/N level of almost 96dB. Signals rarely are maximum swing so the S/N will be less in real world applications.

A correctly sampled audio signal with a sample rate of 16000 samples / second can contain meaningful information up to slightly less than half the sample frequency, see the Nyquist theorem, slightly less than 8000 Hz.

The maximum frequency that the signal can contain very much depends on the filters used in the sampling process. Making very steep analog filters is expensive, so most audio conversions today include a higher initial sampling frequency, followed by a digital filter and downsampling in order to not have artifacts from higher frequencys beeing folded down.

Answer 3

In practice, this does not matter much. All serious work normalizes audio levels. In our code base, there's even some code that runs a nightly check to verify our algorithms are gain-independent.

We recognize that the external format is typically 16 bits, but this does not need to match the internal formats used in transforms. Internally, extra precision can help, especially to avoid rounding errors and clipping. But there's no standard for this; it depends on the application. Translating to Q23.8 fixed point is perfectly reasonable.

Answer 4

librosa does one thing, scipy does another

Actually you are only using scipy.io.wavfile to read in the int16 values. The difference in results comes from the next step y.astype(float32) where y is a NumPy array, a general purpose numeric container, unaware of the fact that audio data is conventionally in the [-1,1] range in floating point format.

Answer 5

The answer is that it depends. The wave file format is a RIFF file which is broken up into chunks. One chunk, "fmt ", is the format chunk describing the format. Within that chunk, the 16-bit word at index 8 within that chunk specifies the format. The only one that I know of that is defined is 1, which is a Linear PCM format. In this mode, specifies the type of sample. 8 bit samples are unsigned, while 16 bit samples are 2's compliment. Regardless, the fact that they are encoded using Linear PCM is probably the information that you need. In particular, the values are linearly spaced. Everything I have seen suggests those are interpreted as voltages to a driver (as opposed to powers or some other unit). The format does not specify an exact voltage, but many systems sort of agreed upon a rough line level around -1V to +1V max with a slew of details surrounding that. Again, those conversions to voltages do not appear in the semantics of the wave file, but it is useful to have some sense of how others interpret it.

I was not able to find out where the zero point for 8 bit samples were, but I would assume it would be 128 for maximum convenience. Actual audio hardware has DC filtering , so it would not matter where the 0 point is, other than that switching between wave files with different 0 points will result in a charactaristic "pop" as it handles the discontinuity.

Answer 6

This is not relevant to the question, but you will get more musically useful results doing spectrograms with a logarithmic frequency axis, not a linear one. The latter fills half of the graph with the top octave, the next quarter with the second-to-top, and all the useful human auditory information is crushed into the bottom cm of the graphic, while a log freq axis devotes the same space to each octave, like the human ear (well, almost).

I think audacity has an option for this, though it's blocky, and sndfile-spectrogram now has a -l option to do this. If you need further help with ways to achieve this, just write.