You can check out the manual for the HP3582A spectrum analyzer or the manual for the SR785 dynamic signal analyzer, which is basically a copy of the former instrument. My guess is that the definition used in these manuals is the closest to what your lecturer is referring to.
In this case, the power spectral density RMS average is found by taking successive averages of the product between the FFT and its complex conjugate (the absolute value), i.e.: $$ \hat{S} = \left< FFT^{*}(\text{windowed time-series segment}) \cdot FFT(\text{windowed time-series segment}) \right> $$
According to the manual:
[...] RMS averaging reduces fluctuations in the data but does not reduce the actual noise floor (squared values never cancel). With a sufficient number of averages, a very good approximation of the actual noise can be obtained.
I belive this should be equivalent to the Bartlett method (periodogram averaging), mentioned by Atul Ingle. The salient point regarding periodogram averaging is the fact that taking the dicrete Fourier transforms (DFT) of a long time-series as a PSD estimate will not lead to a low variance of the PSD estimate. However, segmenting the time-series and taking the average of the DFTs of the segments will reduce the variance of the resulting PSD estimate.
A very good reference with regards to PSD estimation can be found in S. M. Kay and S. L. Marple, "Spectrum analysis—A modern perspective," in Proceedings of the IEEE, vol. 69, no. 11, pp. 1380-1419, Nov. 1981.