The power spectrum is a general term that describes the distribution of power contained in a signal as a function of frequency. From this perspective, we can have a power spectrum that is defined over a discrete set of frequencies (applicable for infinite length periodic signals) or we can have a power spectrum that is defined as a continuous function of frequency (applicable for infinite length aperiodic signals).
In the first case, each discrete component has units of power (W, mW, etc.). In the second case, each point in the continuous spectrum has units of power per frequency (W/Hz, mW/Hz, etc.). In the second case, one must integrate over a band of frequencies to obtain units of power. It is meaningless to talk about the amount of power at a frequency in the second case, one must talk about the amount of power contained in a spectral band (an interval of frequencies) or in several such bands. It is meaningful however to talk about the amount of power spectral density (PSD) at a particular frequency. This is an indicator of how much weight this frequency will contribute to the overall power if included in one of the spectral bands.
This allows us to compare the distribution of power at various frequencies in a signal (as opposed to comparing the power directly). This distribution is defined over a small delta of frequency (delta going to zero in the calculus sense). An alternative to PSD, would be to divide up the signal into chunks of a finite size (say 10 Hz) and compute the power for each bin (one way to obtain this would be to integrate the PSD over each bin). If another person performed a similar experiment and generated their own plot using a larger bin size (say 20 Hz), then the overall shape of both plots would be similar, but the latter plot would seem coarser and its numerical values would be greater. This would make it difficult to compare the two signals. Using power spectral density, the amplitude and bin size ambiguity is eliminated. This allows a fair comparison between
the power distribution of two different signals.
For more information, here is a link to a related Wikipedia article.