As far as equalizer performance and possible limitations I provide two key points below about the span of the equalizer and the number of samples per symbol to use.
The equalizer duration in time is set to match the time duration of the delay spread of the channel (since this is the distortion the equalizer will compensate for). If it is much less than there will be "echoes" (leading or trailing) that will not be properly regrouped into the combined signal, and if it is much longer than you are degrading the signal due to noise enhancement (due to the addition of independent random processes which each coefficient in the equalizer will contain).
Fractionally spaced equalizers are desired to compensate for the full bandwidth of the waveform. For pulse-shaped waveforms where the occupied bandwidth is not much more than the symbol rate, then 2 samples per symbol would be more than sufficient. If the waveform has larger spectral occupancy, then we would follow Nyquist (which is twice the bandwidth NOT twice the highest frequency) to have a sufficient number of samples (sampling rate) to properly capture the full bandwidth of the waveform. Baud rate equalizers (1 sample/symbol) will still work but they will dominate their correction in the center bandwidth of the waveform, and not do very well at the edges due to the aliasing that such sampling would cause. I show this in the graphic below:
As a final point on equalizer performance; the above describes "linear equalizers" which will not perform well in frequency selective channels. This occurs specifically when the delay spread of the channel exceeds the symbol duration (or more specifically for the OP's case, when the inverse of the delay spread is less than the occupied bandwidth of the waveform) as such to cause deep frequency selective nulls within the waveforms bandwidth (and if the bandwidth is wider such as rectangular pulses, this is also more likely to occur). A linear equalizer will cause noise enhancement on these null locations leading to degraded performance (and a non-linear equalizer such as a decision feedback equalizer may be a better choice). This would only occur if there are strong multipath components of similar magnitude and delayed longer than the inverse of the spectral occupancy (to cause deep variation between constructive and destructive interference).
With that in mind and specific to the OP's case of using a rectangular pulse, ultimately in any real-world system of finite energy, the pulse will be bandlimited. Further, when the channel is a shared resource, the metric of spectral occupancy drives the decision as to how much to bandlimit the channel. Also consider if there was no band-limiting (ultimately here in the receiver), the power spectral density of the waveform is decreasing (at rate $1/f^2$) while the noise is at least constant over a much wider frequency range (typically as the noise floor). The receiver will benefit by filtering the waveform regardless as to what was transmitted, and specifically with a matched filter such as to optimize the received SNR (which in this case of a rectangular pulse could be done as an integrate and dump over the symbol period). Such a filter has an equivalent noise bandwidth of $1/T$ where $T$ is the symbol duration. To address the OP's question as to impact of further band-limiting (regardless of equalizer implementation), that could be in comparison to an unlimited pulse assuming the matched filter receiver, and of course its performance in the presence of noise (which is always considered). What you will find that with the proper matched filter- adding more bandwidth (approximating a true rectangular pulse of infinite bandwidth) will always increase the SNR, but you will see as you integrate each additional sidelobe a clear point of diminishing returns-- a tiny fractional increase in SNR at the expense of bandwidth that in order to reap the receiver will also need to handle (with associated power and complexity). Filtering the waveform as the OP has attempted if done incorrectly could then add further inter-symbol interference (ISI) when the impulse response of the filter is non-zero at subsequent symbols, leading to more work for the equalizer. Typically such filtering is done with pulse shaping designed to have zero ISI but excellent filtering by having a long impulse response (long in time is short in frequency) but ensuring that it crosses zero at the ideal sampling location of the subsequent symbols based on their matched filter implementation in the receiver.
Specific to the question of quantifying the error by removing higher frequency components, I made an attempt to compute the actual diminishing returns of SNR benefit vs bandwidth for a perfect rectangular waveform with the ideal matched filter in the receiver. Assuming my thinking is correct, I believe the "processing gain" in the matched filter would be given by the following:
$$G_P(B) = \frac{\int_{-B}^B Sinc^2(\omega T/2)d\omega}{\sqrt{\int_{-B}^B Sinc^4(\omega T/2)d\omega}}$$
Here I am using the non-normalized form as $sinc(x) = \sin(x)/x$ and $B$ is the single-sided bandwidth in radians/sec. I am basically computing the SNR of the matched filter in frequency, and assuming white noise with the numerator representing the weights on the signal components and the denominator representing the weights on the noise.
In dB units of SNR gain this would be $20Log_{10}(PG)$. Graphing this relative to a symbol rate $1/T$ in Hz, we would get the following when normalized to $G_P(B=\infty)$ (To be clear, $B$ is single sided, so the first null of a rectangular waveform of rate $T$ would occur at $1/T$, so plot is normalized to the data rate):
So this is suggesting, if we are not constrained by bandwidth for regulatory or other reasons, and are not concerned with the higher processing rate in the receiver, we can increase achievable the SNR by nearly 1 dB if we extend the bandwidth from the main lobe at $1/T$ out to $10/T$. However also note that wen going from $4/T$ to $10/T$ we only gain about 0.1 dB, hence the consideration of diminishing returns.
