What is the rationale for the two-step process the MP3 format performs, first decomposing the input into 32 subbands of 6/18 samples each then performing MDCT on each subband individually?
Why not simply perform a single MDCT?
- Is this simply due to the non-power-of-two block size MP3 uses (no idea if there is a fast DCT algorithm for that size off the top of my head), or is there some more theoretical reason?
MP3 is a lossy compression which exploits -psychoacoustic masking.
Basically, the human ear has a perception limitations in identifying sounds due to active processing as well as due to non-linear frequency response. Also note, that sound pressure level - the scale that corresponds to perceived loudness is actually logarithmic not linear.
OK - so this only indicates that we should "chop -off" frequencies that are not perceptible to human ear and keep those for which human ear has a high response. this simply can be achieved in 2 ways - either simply apply a filter to the sound and then encode it, or convert the entire signal into frequency domain and drop those frequencies that are on the edge.
NOW: MP3 does something much more than this.
What we see here, is that in the presence of frequency 250Hz, the 175Hz sound is not-audible at all to the human being unless that frequency also has dB level above a certain threshold mask. This frequency is called masking frequency. This is not a dominance of high frequency or low or otherwise. This is simply a function of which frequency produces sound at higher dB (at that time). the threshold of the audibility is also not static but depends on the dB level of the masking frequency. So in order to identify which all frequencies will go un-audible in the presence of other masking frequency one needs to first analyze the signal.
In actual effect the frequency masking effect is continuous in nature - but in practical codec - this is applied over bands.
Hence, we first divide the signal in frequency bands and then identify if any of the frequency bands will go un-audible in the presence of frequency bands which are louder. This is impossible to achieve through linear filtering of any kind! Hence, the filter-bank stage is employed.
The different implementations of filter banks: MP3 uses 2 stage filter bank - PQMF which produces 32 sub-bands followed by MDCT filter-bank which produces 575 bands. This is called hybrid filterbank. In general, MDCT can achieve higher resolution of filter bank which - in general PQMF can't achieve. Hence, Advanced Audio Codec (AAC, the codec usually found in m4a and mp4 files) actually removes the redundant PQMF stage and uses MDCT event at a higher resolution. See Ref 3.
However, so the question is why is MP3 stuck to 2 stage filter bank? As it turns out, the reason is more to do with compatibility with MPEG Layer I and Layer II. As indicated in the document here MP3 AND AAC EXPLAINED Ref 4 right from Fraunhofer, it says the real reason why MP3 remain stuck with 2 stage filter bank is due to backward compatibility with MPG 1 Layer I & II. From the paper (3.3.1):
The polyphase filterbank has the purpose of making Layer-3 more similar to Layer-1 and Layer-2.
Ref : Wikipedia - Psychoacoustics
Ref : Theory behind MP3
Ref : Modified Discrete Cosine Transform-Its Implications for Audio Coding and Error Concealment
Ref : MP3 AND AAC EXPLAINED also available here.
Newer transform-based codecs do simply perform MDCT-based coding. MP3 carried over the filterbank structure from MP2 (MPEG I layer 2) as a historical artifact.
Separating the input signal into subbands allows to use different MDCT parameters for each band.
These parameters are determined by the psychoacoustic model, which determines, for each subband, the window length for the DCT and quantization steps for the resulting coefficients. See, for example, slide 9 of this summary.
Performing only one DCT for the whole signal would not allow the psychoacoustic model to adjust the admissible quantization noise and window in each subband, which is the basis for data rate compression.
Further compression is then performed by source coding (Huffman).