In the video that OP linked, as well as here (pg. 22), it is derived that the variance of power is proportional to (and in the linked ref, equal to) the square of the mean power, i.e.
var[ P(w)] = \bar P^2(w)
Distributions that follow the general relationship: $$var[ P(w)] =\alpha \bar P ^x(w)$$ are of the Tweedie family. In this case, the gamma distribution satisfies x=2.
A quick Python simulation with normally distributed noise where $$\mu=0, \sigma=6$$
corroborates this. The left plot is a histogram of variance computed over 4000 non-overlapping 1000-point time windows. Middle plot are the distributions of absolute Fourier amplitude, where each faint trace is the distribution at a single frequency (1000-point FFT). The right plot is the same, but for power (i.e. amplitude squared). The mean of those distributions (not shown) is indeed 0.036 (which is the signal variance over the number of FFT points, 6^2/1000). We see that the distributions are heavily skewed, and resemble the form of gamma distributions, but that's about as much as I can say.
Therefore, I'm venturing a guess that the noise distribution in the frequency domain follows the gamma distribution. I realize that this is without any formal derivation, and I would love to see a derivation of this as I am also interested in this problem.
Lastly, with reference to the above (Maximilian's) answer: please correct me if I'm misinterpreting your response, but the noise in the frequency domain cannot be Gaussian distributed with zero mean, right? Because the mean is defined as the mean power at that frequency, which is proportional to the variance of the signal in time domain. Sorry, I don't have enough points to comment directly below the response.
Edit: the Fourier coefficient themselves, i.e. real and imaginary components, are distributed normally with zero mean and standard deviation scaled to the total signal power, but not the signal magnitude.