A convenient approach for studying the response of a stable LTI system with impulse response $h(t)$ to a WSS stochastic input $X(t)$ is to look at the power spectral density (PSD) of the output $Y(t)$ as $S_Y(f) = \vert H(f) \vert^2 S_X(f)$, where $H(f)$ is the Fourier transform of $h(t)$ and $S_X(f)$ and $S_Y(f)$ are the PSDs of the input and output, respectively. For simplicity, let's assume that $X(t)$ is zero-mean.
Now if I have an unstable LTI system for which $h(t)$ may not even have a Fourier transform, this method naturally won't work anymore. Is there a similar straightforward approach for looking at the statistical properties of the output of an unstable system?
The only approach I can think of is to directly look at the convolution integral $R_Y(\tau) = h(\tau) \star R_X(\tau) \star h(-\tau)$ but that's not as intuitive as the frequency-domain analysis we have for stable systems.