OFDM use a pulse shaping filter or not?

I've heard in various literature that band limited channels use pulse shaping filters. By the way, In OFDM it use a cyclic prefix to overcome ISI, so I wonder if OFDM does not use pulse shaping filter.

The purpose of pulse shaping filters is not to overcome ISI as is implied in the OP's question. The only reason for using a pulse shaping filter is spectral efficiency, and in the process ISI can be introduced if not done properly. In order to limit bandwidth, pulse shaping must extend the time domain response for each pulse beyond a symbol boundary, but can be done such that the response passes through zero at the symbol decision location of ever other pulse (Nyquist shaping filter, raised-cosine filter etc).

A rectangular pulse in time is a Sinc function in frequency, with the main lobes of the Sinc that roll-off at only 1/f (20 dB/decade) as shown by the blue spectrum for a single-carrier rectangular pulse modulation in the plot below. The red spectrum shows the significant advantage is spectral containment by pulse shaping:

Notice this about the plot above to see intuitively why we are less concerned with using pulse shaping for OFDM (or when we may want to consider its use). The blue spectrum above is rolling off at 1/f where f is proportional to the symbol rate. If we reduce the symbol rate for a single carrier by a factor $$N$$, we equivalently increase the rate of frequency rolloff by that same factor. OFDM converts a single carrier modulation into $$N$$ sub-carriers, each representative of the spectrum above in blue, each with much reduced bandwidth individually and therefore overall the resulting waveform extends much less in frequency beyond its primary occupied bandwidth as given by $$NR$$ where $$R$$ is the symbol rate of each of the sub-carriers.

So to recap that with the plot of the blue spectrum above as an example: Here we see the result of a 1 MHz symbol rate (1 MSymbol/sec) single-carrier modulation, and notice within the range of the plot the spectrum rolls off to about -35 dB at 5 MHz offset. Consider if we instead sent the same amount of data by using 64 carriers, all reduced in rate by 1 MHz/64, and spaced in frequency by the same (with is orthogonal carrier spacing). The resulting spectrum would dominantly be in a 1 MHz span of frequency, with the carriers on the ends contributing the most to the spectrum going beyond that range, and specifically we would expect about -35 dBc at a 5 MHz/64 = 78.125 KHz offset. That, without any additional pulse shaping, wouldn't appear very different from the red plot above (which is a single carrier pulse-shaped waveform)! If we wanted to make the spectrum even tighter (with the diminishing return of benefit), we could certainly apply pulse shaping to reduce the adjacent leakage terms accordingly.

• Interesting point. However, we don't take a mono-carrier bandwidth and divide it by a number of subcarriers, but rather keep a certain subcarrier-spacing and multiply it with a very important number of subcarriers. Indeed, I often see OFDM systems with large bandwidth. – AlexTP Apr 15 at 7:51
• Your point is convincing though, but do you know why in practice, for example in LTE, we need to reserve about 40% subcarriers at the borders of bandwidth, and also apply spectrum masks to avoid out-of-band interference? – AlexTP Apr 15 at 7:51
• @AlexTP it is effectively the same: for a given data set I can send it with a single carrier or I can effectively split the data up amongst many lower data rate carriers all evenly spaced and transmit in parallel. The pulse shaping would be just as effective but scaled down proportionally, so to the extent pulse shaping is needed in a single carrier modulation that is much reduced in OFDM – Dan Boschen Apr 15 at 11:07

OFDM itself doesn't use a pulse shaping filter. The spectral shape is inherent.

There are systems which apply a pulse shaping filter to the OFDM signal, but I'd call this "OFDM plus a pulse shaping filter", not just "OFDM".