I'm trying to build a "perfect" low-pass filter as an exercise to test that I've understood some basics in signal processing.
I've computed the required length of a sinc function such that at either end the value is 0 for a given integer bit depth of output. For example, 16-bits would require a filter that is 41722 taps.
I'm then applying this filter to a frequency sweep file sampled at 48kHz, with Fc set at 0.5, thus it should act as a low-pass filter with the cutoff frequency at 12kHz.
When I feed this filter some data I get unexpected results. For the 16-bit filter, the low-pass effect is clear, but there is data beyond 12kHz as well as other corruptions. For a 20-bit filter, 667543 taps, the output is totally wrong.
In the 16-bit case, applying a blackman window to the sinc function improves the output and makes it look correct, but this doesn't help in the 20-bit case.
I've attached a couple of images to illustrate the problem. I'm fairly certain that my sinc function is generating correct data. I'm using
vDSP_convD to apply the filter (I've also tested with a handwritten function, with no change.)
Am I misunderstanding how a sinc function can be used as a low-pass filter, or am I running into other problems?