First, there are many ways to do your task. Depending on your background, some might be easier than others for you to understand and use.
If you are familiar with signal processing, you might construct a noise model and based on this noise model to detect whether or not a given signal is of pure noise. This type of method often involves hypothesis tests, and test statistics. Based on these things, it rejects or fails to reject a given sample accordingly. This is a quite traditional way to solve your problem.
If you really donot care about details here, you might complete ignore signal and audio. Purely consider what you have are different data streams, and you can use all kind of machine learning methods to differentiate two classes of things, in your case, one is noise and the other is non-noise. If you are going to adopt this approach, python sklearn package will be your good friend with many great tutorials teaching 0-level people to do machine learning.
Finally, let me remind you that one big difference of the later approach from the former one is that you need prepare labels of each signal sample. Otherwise, ML algorithms will have a better chance to differentiate these two classes when you have labels and many samples. Anyhow, both approaches are belonging to the pattern recognition family, and thus they are definitely related to each other. For example, you might view a probabilistic noise model is somewhat a naive bayes classifier.