According to Nyquist, to transmit R symbols/sec without ISI, the minimum bandwidth required is R/2 Hz, and the sinc pulse shape satisfies this condition.
1. This is a question investigated by many researchers decades ago. They discovered that the bandwidth limitation of R/2 Hz is not a fundamental limit set by nature. It is just a criterion if we don't want to do any further processing at the Rx (that also doesn't mean that we can infinitely break this limit).
As long as we are willing to pay the cost in terms of Rx complexity, we can go higher than the rate R within the same bandwidth R/2. Mazo found that 25% faster than the rate R can be accommodated without any penalty other than the higher computational complexity. Pulse shapes are still the same but they are packed more closely.
It was mostly an academic thing before the advent of turbo codes. After the turbo codes, iterative receivers became a reality and then people started implementing turbo equalization where the decoder and the equalizer exchange extrinsic information to improve their belief of each bit.
Then, incorporating a higher symbol rate became possible and this is known as Faster than Nyquist (FTN) signaling. The idea was further enhanced to incorporate frequency domain packing as well (think OFDM) and now people break the limits in both dimensions and still recover the information.
Coming back to your question, no we don't strictly need to obey this limit as long as we are aware of the price we have to pay.
2. I assume that you are asking about the system cumulative impulse response (and not the transfer function). When employing trellises, Viterbi algorithms and iterative processing, the concept of a pulse shape does not go along that far. The samples affected with inter-symbol interference are taken and this interference is canceld out in some fashion, e.g., sequence estimation or iterative receivers.
