After not having dealt with digital filters for a long time, I have now been playing around with filter design in octave and I am observing a behavior which I do not understand.
After designing a low-pass Chebychev filter with
cheby1(1,3,0.4) and plot its frequency response, it can be seen that the -3dB point is exactly at the specified cutoff frequency.
However, when I test the filter with an input wave
x=cos(2*pi*cutoff*t), using the filter function
y=filter(b,a,x), the amplitude of the output signal can get significantly lower than the expected $0.707$, $0.63$ for this example in particular.
It seems that the output of the filter better approximates the plotted frequency response for cutoff frequencies either close to Nyquist frequency, or close to $0$, but as it gets closer to the middle point between these two, the output steps away from the expected attenuation.
What is the reason behind this?