Currently I am writing a script that will look at the time/size = compression rate. Anything that is below 300kb/s I want to delete.
That's a very bad idea.
Just because something compresses well, because it fits the signal model of the compressor well, doesn't mean it's bad quality. Usually, quite the contrary.
You give an example in your comments yourself:
Im talking about tracks that have lot of clipping
Clipping is a nonlinearity, introducing a multitude of tones into the spectrum that shouldn't be there. Since psychoacoustic models and thus lossy compressors usually need to quantize the spectrum in some way, having clipping in your audio means harder to compress audio meaning larger file size.
Generally, if something is hard to compress, a variable-rate codec will increase the file size. If something is easy to compress without much loss, then the file size will stay down. I'm pretty sure you can record a pretty perfect tuning fork tone in high quality and compress it very nicely.
Also, unless we're talking early MP3 encoders and such, modern codecs above ca. 160 kb/s are really hard for a human to tell from the uncompressed audio. That's the whole idea of having a good lossy compressor: Just lose the information that's irrelevant.
Note that I'm not saying you can't generally say that for example an MP3 with 64 kb/s will sound bad in almost all cases. It's just that you don't need any intelligence to do that, but can often directly read that from the file's metadata.
Like looking at a spectral analysis to determine what is low vs high quality.
Nope, you're looking at the reconstructed signal after decompression. Unless you have extensive knowledge of the spectrum of your original audio, there's little information about audible losses in that. Again, both musicians, sound engineers and audio codecs strive to do the same: produce sound that sounds good to the human ear.
Also, remember that clipping (or soft clipping, and other similar nonlinearities), as the example you chose, is actually a pretty commonly used tool in production of music these days. Who are you to tell someone that the recording of their neighbor's cat walking on a keyboard is quality-wise better than let's say a Kraftwerk song that uses clipping intentionally? How is a crisp-sounding KPop-girlband song with strongly emphasized treble quality-wise superior to the muddy sound of the 1968 original release of Jimi Hendrix' All along the watchtower? There's whole musical genres that depend on sound mechanics that would signify a lack of recording quality in others. Even within the same genre, quality can't simply be deducted from a rigid description of the signal – compare Shostakovich's The Bolt to other ballet suites of that era.
"Good audio quality" in the end is an extremely subjective matter, and I'm afraid without a large-scale database of manually tagged "good" and "bad" examples according to your perception, you won't be able to implement any classifier, be it through neural nets or through more classic statistical approaches.