3
$\begingroup$

enter image description hereThe title is the question. To make this concrete, assume I have a standard 16-bit single channel wav file, with (although it doesn't matter) a 16000 Hz sampling rate.

Each sample magnitude will be a 16-bit value.

Is there a specified semantics or interpretation of these values, not relative to any physical value, but relative to the numerical systems of modern computers? I.e., are those numbers to be interpreted as 16-bit twos-complement, or scaled into single precision floats between -1 and 1, etc?

At the highest level, I recognize this is a little pointless: They're arbitrary scale.

But at a practical level, in the python world, librosa does one thing, scipy does another, and converting those different things into a spectrogram or analyzing with a vocoder can yield different results.

For skeptics, invite you to try the following:

import scipy
import librosa.display

np_eps        = np.finfo('f').eps
np_max        = np.finfo('f').max

x, sr = librosa.load(file, sr=16000)

plt.subplot(2, 2, 1)
D = librosa.amplitude_to_db(np.abs(librosa.stft(x)), ref=1.0)
librosa.display.specshow(D, y_axis='linear')
plt.colorbar(format='%+2.0f dB')

plt.subplot(2, 2, 2)
plt.plot(x)

sr, y = scipy.io.wavfile.read(file)
y = y.astype(float32)
plt.subplot(2, 2, 3)
D = librosa.amplitude_to_db(np.abs(librosa.stft(y)), ref=1.0)
librosa.display.specshow(D, y_axis='linear')
plt.colorbar(format='%+2.0f dB')

plt.subplot(2, 2, 4)
plt.plot(y)

Do not neglect the extra argument to librosa, forcing the sampling rate; in my case, the file I am using is from the Arctic corpus and is sampled at 16 KHz. Do not neglect the colorbars, which will show you the very obvious shift of 90 dB from one to the other. This should not be surprising, since 20 * log10 (32768) is approximately 90.3.

I assure you, gentle readers, this will have consequences further down the pipeline of speech vocoders, or neural networks (where normalization techniques might want you to reference to numpy epsilon to avoid negative numbers) or both.

$\endgroup$
  • $\begingroup$ To put what @Fat32 said in other words: the samples may be processed by any number of different programs before being written to the WAV file. For example, they may be normalized, resampled, filtered, etc. The WAV file itself does not impose any semantics on the data it stores. $\endgroup$ – MBaz Nov 17 at 22:40
  • $\begingroup$ "librosa does one thing, scipy does another, and converting those different things into a spectrogram or analyzing with a vocoder can yield different results." I don't believe that's true. Can you give an example? Maybe your real problem is that you're doing the spectrogram wrong? $\endgroup$ – endolith Nov 18 at 1:58
  • $\begingroup$ @endolith How could it any other way? That librosa and scipy do different things shouldn't be in dispute, the former is scaled to [-1,1] and the other isn't. Given that, why on earth would you expect a spectrogram algorithm to yield identical results on different inputs? $\endgroup$ – Novak Nov 18 at 4:26
  • 1
    $\begingroup$ I don't understand this question. The premise is that the audio is " standard" 16 bits, that is to say PCM encoded, signed 16 bits integers. If it's 16 bits, why ask if it's 32 bits? It obviously isn't. Nor is it a float between -1 and 1. It's an integer between -32768 and +32767. We might argue about -32768, that's about it. $\endgroup$ – MSalters Nov 18 at 7:18
  • $\begingroup$ @MSalters you caught a typo (now fixed.) Where I referenced two complement 32-bit, I meant to say 16-bit. The question is premised on the observation that different software packages do different things, and I do not understand why-- is one way definitively correct, or is there an unfortunate ambiguity in the specifications? $\endgroup$ – Novak Nov 18 at 8:49
2
$\begingroup$

I don't know much about this semantics? of WAV files but their numerical format is the following. (assuming mono)

  • Given a recording with 8-bit per sample precision, then those samples are unsigned integers taking values between $0$ and $255$. Due to being unsigned, to represent negative values, there is a bias of $128$, and the sample values are actually interpreted to be between $-128$ and $127$. In this case a value of $128$ represents silence, maximum value $255$ corresponds to an input full scale positive audio waveform and $0$ represents neagitve full scale input waveform.

  • For $16$-bit WAV files, you have $16$-bit signed integers that range from $-32768$ to $32767$ and $0$ represents silence. Again, $32767$ represents full scale audio input without scaling.

Note that here I refer to a microphone recording through an ADC system which converts analog audio input in the standard range of about $1$ V rms into WAV samples. Please check this 1-V rms for being something higher, about 1.5 Volts or else depending on particular implementation or some specific audio standard. In any case, the analog circuitry is responsible for transmitting a full scale audio to the ADC whan a 1V (or 1.5V) rms wave is present at the input and then the conversion is performed according to the above setting when WAV recording is performed.

$\endgroup$
  • 1
    $\begingroup$ 8 bits per sample is often G.711 encoding (A-law or u-law). Your assumptions might be flawed there. Also, the "1V rms" is rather specific (Line-level voltages) and is entirely wrong for microphones. Those work at millivolt ranges. $\endgroup$ – MSalters Nov 18 at 10:29
  • 2
    $\begingroup$ @MSalters as far as I know, Microsoft Windows proprietary wave format file .wav does not require or specify any companders (A-law ou mu-law) at 8 bits per sample. And furthermore that's not something I've observed when using them? May be for Microsoft Telephony API they would implement those companders via additional processing. And for the voltage level, I was not referring to microphone capsule output, rather ADC input; in between there are amplifiers which should convert microphone output voltage into fullscale ADC input voltage under nominal operating conditions. $\endgroup$ – Fat32 Nov 18 at 13:14
2
$\begingroup$

The semantics for a sampled audio signal is very simple. Each sample represents an amplitude, each sample is done at a specific time.

If you create a signal using a microphone, the amplitude is related to the pressure as measured by the microphone diaphragm. The sampling process will introduce time in the equation.

In the question, there are two sets of diagrams. The right hand diagrams shows the signal amplitude although the individual samples are difficult to see there (you may magnify). You might see it as the deflection a loudspeaker cone would do.

The left hand diagrams is the result of a Fourier transform, where the signal is presented as both a frequency and a "loudness". You could see this as a representation closer to what our mind perceives when listening.

The 16 bit values in the wav file are amplitude values, encoded as signed 16 bit numbers. The amplitude can go from -32768 to +32767. The value can often better be interpreted as (almost) -1.0 to +1.0, but this is an interpretation.

In audio we may interpret a maximum signal, swinging from -1 to +1 as having a loudness of 0dB. Smaller signals would have negative values, say -20dB.

Audio signals tend to have a mean amplitude of zero. This comes from the high pass filter of the audio circuits in front of the AD-converter. Most often we do not want to include signals below something like 20 Hertz. A mean value outside of zero would indicate a DC signal, frequency 0Hz.

The number of bits in the samples, here 16, is related to the maximum Signal to Noise ratio, known as S/N, that the signal theoretically can have. Very simplified, each bit allows about 6dB of possible S/N. A maximum swing signal, going from -1 to +1 would then approach a S/N level of almost 96dB. Signals rarely are maximum swing so the S/N will be less in real world applications.

A correctly sampled audio signal with a sample rate of 16000 samples / second can contain meaningful information up to slightly less than half the sample frequency, see the Nyquist theorem, slightly less than 8000 Hz.

The maximum frequency that the signal can contain very much depends on the filters used in the sampling process. Making very steep analog filters is expensive, so most audio conversions today include a higher initial sampling frequency, followed by a digital filter and downsampling in order to not have artifacts from higher frequencys beeing folded down.

$\endgroup$
2
$\begingroup$

In practice, this does not matter much. All serious work normalizes audio levels. In our code base, there's even some code that runs a nightly check to verify our algorithms are gain-independent.

We recognize that the external format is typically 16 bits, but this does not need to match the internal formats used in transforms. Internally, extra precision can help, especially to avoid rounding errors and clipping. But there's no standard for this; it depends on the application. Translating to Q23.8 fixed point is perfectly reasonable.

$\endgroup$
1
$\begingroup$

librosa does one thing, scipy does another

Actually you are only using scipy.io.wavfile to read in the int16 values. The difference in results comes from the next step y.astype(float32) where y is a NumPy array, a general purpose numeric container, unaware of the fact that audio data is conventionally in the [-1,1] range in floating point format.

$\endgroup$
1
$\begingroup$

The answer is that it depends. The wave file format is a RIFF file which is broken up into chunks. One chunk, "fmt ", is the format chunk describing the format. Within that chunk, the 16-bit word at index 8 within that chunk specifies the format. The only one that I know of that is defined is 1, which is a Linear PCM format. In this mode, specifies the type of sample. 8 bit samples are unsigned, while 16 bit samples are 2's compliment. Regardless, the fact that they are encoded using Linear PCM is probably the information that you need. In particular, the values are linearly spaced. Everything I have seen suggests those are interpreted as voltages to a driver (as opposed to powers or some other unit). The format does not specify an exact voltage, but many systems sort of agreed upon a rough line level around -1V to +1V max with a slew of details surrounding that. Again, those conversions to voltages do not appear in the semantics of the wave file, but it is useful to have some sense of how others interpret it.

I was not able to find out where the zero point for 8 bit samples were, but I would assume it would be 128 for maximum convenience. Actual audio hardware has DC filtering , so it would not matter where the 0 point is, other than that switching between wave files with different 0 points will result in a charactaristic "pop" as it handles the discontinuity.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.