Wouldn't this be equivalent to discarding all those frequencies in the input layer?
No, it won't for 2 facts:
Just like a CNN isn't a linear regression due to having non linear function in between. So the Activation Layers on the frequency domain mean things are not propagated linearly in the forward pass.
The filters are adaptive (Learned). Namely each ...