Note that this illustration is just that, an illustration: The spectrum on the right can't be explained by the time-domain signal excerpt on the left alone. What's shown on the right is the spectrum that's typical for a tone with some phase noise, no matter where that comes from. But since frequency is the time derivative of phase: no, if you get something like the right spectrum, you don't only have a tone of a single, constant, frequency.
Why isn't this a bar when the frequency of the signal clearly doesn't vary?
Because only periodic signals have line spectra. Your OFDM signal isn't periodic (for longer than an OFDM symbol), so it can't have a line spectrum.
Since any information transmission requires change of some parameters of a wave to actually contain any information, no information-carrying signal can have (with sufficiently long observation) have a line spectrum. End of story!
You'll notice that when reading up on pulse shaping, for modulations¹ where there's no DC offset², the pulse shaping defines the spectral shape.
In OFDM, the pulse shape is a rectangle the length of the OFDM signal. Hence, the sinc-shaped spectrum. (and the trick about OFDM is that the zeros of the individual sincs happen to fall exactly on the maxima of the other sincs, so that they don't interfere, hence the "O" in OFDM.)
¹ no matter whether we're considering a single-carrier system or just one subcarrier of an OFDM system
² i.e. all practical digital modulations aside from a few power detection-based specialities.
