FFT magnitude as a function of frequency resolution

My learning research on FFTs has identified that (for example) with a FFT frequency resolution of 8 Hz, the magnitude of the FFT spectral lines (computed as the absolute value of the complex numbers returned by the FFT function) should be higher than if a FFT frequency resolution of 4 Hz is used (twice the number of samples per FFT) on the same (IQ) sample data. I see evidence of this when using a Spectrum Analyser receiving broadcast FM signals and also in SDR software when doing the same (both when changing the RBW). But, when using the MatLab FFT function (on broadcast FM IQ data) the opposite is what I see - i.e the higher the FFT frequency resolution (lower number of samples per FFT), the lower the scalar magnitude of the FFT complex values. What I am I doing wrong / not understanding?

Bottom Line: From the O.P.'s description, the FFTs reported by the Spectrum Analyzer and SDR are scaled by dividing by the number of samples (and possibly additional scaling factors as explained in the links at bottom if windowing is used), which is reasonable to do. The FFT's returned by MATLAB simply need to scaled by dividing by the total number of samples to match this.

Details:

The frequency resolution as reported by a spectrum analyzer (RBW) will indeed result in higher signals for wider resolution bandwidths when the signal energy is spread wider than the bandwidth. This makes sense as the analyzer is reported the total power within the bandwidth of a filter, centered on any particular frequency, and we observe this power as the filter effectively sweeps over a range of frequencies, or equivalently done with a filter bank where we observe the total power within the bandwidth of each filter.

The DFT (which the FFT as an algorithm computes) also has a resolution bandwidth, and is equivalently a bank of filters with each filter centered on each bin center in the FFT. The resolution bandwidth is also commonly called the equivalent noise bandwidth: the DFT filter without further windowing has an approximate Sinc function magnitude response (and precisely the "Dirichlet Kernel" which is an aliased Sinc function), and the equivalent noise bandwidth (EQNBW) is the bandwidth of a brick filter that would report the same total integrated power as the total integrated power under the Sinc function for the case of white noise. In the case of an unwindowed DFT, the EQNBW is one bin, and this only gets wider if we use windowing (which also improves the DFT as a filter for non- white noise cases).

That said, this is exactly the result the OP is expecting: for a tighter RBW (or EQNBW), the total power in each bin should reduce under the conditions of signal spectrums with power that is spread over multiple bins, and this is exactly the result we will see. For this reason the "DFT Noise Floor" as observed when we take the DFT of a white noise process, will go down as we increase the number of bins in the DFT.

The FFT reported by MATLAB however also includes a "DFT Gain" as MATLAB (as well as Octave and Python scipy.signal) provide a non-scaled DFT as the result. When we wish to see a "Spectrum Analyzer" result with an output power level consistent with the power in a particular resolution bandwidth, the FFT result needs to be properly scaled which includes dividing by the total number of points.

The FFT magnitude is simply given by the assumed operations for computing the DFT (if it scaled or not), either approach can be taken and should be defined. (The FFT is an algorithm for efficiently computing the DFT).

For example, an non-scaled DFT is just the sum of the products and given as:

$$X(k) = \sum_{n=0}^N x[n] W_N^{nk}$$

And would result in a scaling by $$N$$ for each of the $$N$$ dft outputs.

And similarly the scaled or normalized DFT is given as:

$$X(k) = \frac{1}{N}\sum_{n=0}^N x[n] W_N^{nk}$$

Where above $$W_N$$ is the "DFT twiddle factor" and given as $$W_N = e^{-j2\pi/N}$$ and all the integer exponents of $$W_N$$ are the Nth roots of unity.

To see this simply, consider the first bin with $$k=0$$ which is the DC bin, and with a DC input in time of all ones for each sample. For that case all the twiddle factors are equal to one and we see that the result sums to $$N$$ for the first case and to $$1$$ for the second.

Note that "Frequency Resolution" is another topic detailed here and is a function of the windowing used if any, and if we are measuring signals with power that occupies only one bin, or is spread across multiple bins, in which case different scaling factors need to be provided.

Other posts related to spectrum analyzers and FFT spectral analysis that may be of interest:

How to reprocess SDR data to be equal to that captured with a higher RBW

Radar terminology - Specific meaning of "video signal"

• Many thanks for your helpful response. If I understand correctly, it is necessary to divide the absolute value of the raw output of the MatLab FFT function by the number of FFT bins, N, in order to observe the effect described on multiple web sites where a larger RBW gives rise to larger 'FFT' spectral line magnitudes. I.e. the 'FFT' that the web sites are talking about is actually the (FFT/N)? If so, that's a relief; when I divide the absolute value of the raw MatLab FFT by N with my data, larger RBW values do give rise to larger spectral line magnitudes
– Rob
Commented Apr 4, 2022 at 8:06
• Following on from this, I've seen web sites explaining the conversion from FFT to PSD saying it is necessary to square the absolute value of the FFT and DIVIDE by the RBW. Would they also be referring to 'FFT' as (MatLab FFT / N)?
– Rob
Commented Apr 4, 2022 at 8:07
• Yes in the simplest case that the sampling rate is 1 (which refers to normalized frequency given as $f/f_s$ where $f$ is the frequency and $f_s$ is the sampling rate), and no further windowing other than selection (rectangular window) is done. Then the RBW is $1/N$ in units of normalized frequency. But as detailed in my linked post in the answer, if we window (as commonly done) then this will increase the resolution bandwidth so we also divide by that factor in that case. Commented Apr 4, 2022 at 11:15
• Thanks - yes I've not been applying any specific windowing in my tests - so rectangular. The calculations all add up now. I found some on-line MatLab code which was calculating PSD by dividing the square of the absolute value of the raw MatLab FFT by (Fs*N), which is algebraically identical to calculating the square of the absolute value of: (raw MatLab FFT / N) and dividing the result by the RBW. :-)
– Rob
Commented Apr 4, 2022 at 18:37
• Apologies; I've just realised that I didn't previously get as far as reading your post at the link 'about PSD scailing factor'. My head was spinning at that time. I see you'd already done the algebra to equate the MatLab code to the standard form of dividing by RBW! :-)
– Rob
Commented Apr 4, 2022 at 21:12