dBFS scaling and spectrum

Question

I am trying to replicate the spetrum as given in Audacity and other commercial software... I am getting crazy with the correct scaling needed in order to have dBFS (with reference to a full scale sine wave).

Suppose $x_t$ is the PCM sample at $k$ bit precision, first of all I rescale the PCM samples

$$ s_t = \frac{x_t}{ \frac{ 2^{k-1} }{ \sqrt{2} } } $$

Suppose now $w_t$ is some window function, I compute the FFT of of size N of the product $s_t w_t$.

Let $P_1, P_2, \ldots, P_N$ be the FFT complex values. Since I am only interested in one-side spectrum, I just retain the fist half of the sequence. Moreover suppose for simplicity that the time series matches the window size $N$.

Taking

$$ p_i = 2\left| \frac{P[i]}{N}\right|^2 $$

for $i=1,2,\ldots, N/2-1$ should give the power spectrum values. Then one would take $10\log_{10}(p_i)$ to get dBFS.

But I think that in order to express dBFS one needs to scale the $p_i$ in some way taking into account the effect of the window. In fact I produced a pure 1KHz sine wave then I computed my spectrum and compared it with that obtained with Audacity, Adobe Audition etc. My FFT stays below by few dBFS so I think it's really a problem of scaling.

I hope somebody knows if there is a standard ways of doing that!

Regards Pierre

Answer 1

Here's a paper with a pretty thorough explanation of proper scaling in order to obtain the power spectrum or power spectral density. Essentially N should be the ENBW or the sum of the window squared.

Answer 2

ENBW = equivalent noise bandwidth. A rectangular window with values $\frac {1}{N}$ (one might say this is the no-window case) has an equivalent noise bandwidth of $1$. Look for fred harris's (he doesn't capitalize his name) paper titled On the use of Windows for Harmonic Analysis with the Discrete Fourier Transform. It's a classic and should explain what you're looking for.