# What is the difference between pulse shaping and equalizers when used to cancel Inter-symbol Interference (ISI)?

According to Nyquist, to transmit R symbols/sec without ISI, the minimum bandwidth required is R/2 Hz, and the sinc pulse shape satisfies this condition.

1. If we use equalizers instead of pulse shaping, do we need to keep this condition ( minimum R/2 Hz to transmit R symbols/sec)?
2. If we use equalizers, should the total system transfer function be of the sinc function shape, thus equivalently achieve the Nyquist condition?
• you don't use equalizers instead of pulse shaping. In a frequency-selective channel, you still need both. Aug 11, 2018 at 21:23

1. This is a question investigated by many researchers decades ago. They discovered that the bandwidth limitation of R/2 Hz is not a fundamental limit set by nature. It is just a criterion if we don't want to do any further processing at the Rx (that also doesn't mean that we can infinitely break this limit).

As long as we are willing to pay the cost in terms of Rx complexity, we can go higher than the rate R within the same bandwidth R/2. Mazo found that 25% faster than the rate R can be accommodated without any penalty other than the higher computational complexity. Pulse shapes are still the same but they are packed more closely.

It was mostly an academic thing before the advent of turbo codes. After the turbo codes, iterative receivers became a reality and then people started implementing turbo equalization where the decoder and the equalizer exchange extrinsic information to improve their belief of each bit.

Then, incorporating a higher symbol rate became possible and this is known as Faster than Nyquist (FTN) signaling. The idea was further enhanced to incorporate frequency domain packing as well (think OFDM) and now people break the limits in both dimensions and still recover the information.

Coming back to your question, no we don't strictly need to obey this limit as long as we are aware of the price we have to pay.

2. I assume that you are asking about the system cumulative impulse response (and not the transfer function). When employing trellises, Viterbi algorithms and iterative processing, the concept of a pulse shape does not go along that far. The samples affected with inter-symbol interference are taken and this interference is canceld out in some fashion, e.g., sequence estimation or iterative receivers.

• The Mazo Limit of 25% only holds for BPSK modulation. As soon as R/2>B, the transmit filters form a redundant frame, i.e. two different transmit data sequences can generate the same transmit signal, hence wont be able to be detected at RX. However, if you restrict to only discrete TX symbols (as is commonly done), one can indeed increase the TX rate above Nyquist. Though, note that the amount of possible speed increase depends on the number of bits per symbol. Aug 12, 2018 at 5:38
• @MaximilianMatthé it would be great if you could extend your comment to a full answer. Aug 12, 2018 at 9:55