Without comprehensively looking at you data, nothing more than speculation can be offered here.
I’m not particularly available but others are probably more eager for a challenge.
not too old statistics books will caution against using estimates of $\sigma$ because they are very sensitive to outliers. See for example:
Wilcox, Rand R. Fundamentals of modern statistical methods:
Substantially improving power and accuracy. Springer Science &
Business Media, 2010.
If your 5.E6 signals are files a two pass forward/backward scheme would make things considerably easier. You avoid the lag that windows introduce as well as getting estimates of average intensity. Even if real time, a fixed lag will improve performance. Your results hint at window lag on the back side of your transients.
There is always a performance complexity trade off but order-statistics are more dependable and if you use a priority queue, it isn’t anywhere near doing a sort every new sample.
In Rabiner’s speech processing book, he follows a median filter with a linear filter to detect quiet intervals.
Rabiner, Lawrence R., and Biing-Hwang Juang. Fundamentals of speech
recognition. Vol. 14. Englewood Cliffs: PTR Prentice Hall, 1993.
You can also go with a run length scheme or an M out of N test on absolute amplitude.
You haven’t said too much about your data but there will be clues.
Finally, the hardest part is defining your metrics. Processing data without having some understanding of your performance is not usually useful. Is it better when low variance intervals are more accurately determined or the converse, Your examples show more high intensity than quiet periods. Running a STFT on some of your data to see if there are spectral changes is worth the effort.
Another thought is to pursue the transition regions. Finding a set of thresholds (or features ) on jumps places less emphasis on the absolute levels of a particular file.
Wavelets often do well in change detection.
In summary, what you have done so far isn't insane and probably reasonable but you should try some other things as well. Good and fast is not impossible but there is a trade off.