I have a dataset of quarter second 16 bit mono PCM audio clips, each clip labelled as speech or nonspeech. I am trying to estimate the quality of the labelling by randomly sampling a few of them and listening to the clips. How do I detect if the samples are labelled correctly by hearing them?
That's really NOT a signal processing question, so I will interpret it as "what signal properties can be used to perceptually identify speech in very short recordings?".
A quarter second is quite short. It accommodates on average maybe 2 syllables or 3-4 phonemes. And that's only on average: one of the most common sounds in speech is silence which can easily last for more than a quarter second.
You first need to define precisely what do you mean by "speech". Let's say you take a recording of an audio book and chop it up into 1/4 second long snippets. Are all of those snippets supposed to be speech (since they came from a real of speech recording)? Even the silence ones ? How about the ones that contain 0.2s of silence and 0.05s of a partial phoneme? How about the ones that are half silence half speech ?
Next I would take a look at the non-speech samples. If it's not speech then what is it? Noise, environmental sounds, traffic, music or movies, silence, typical residential background, all of the above, etc? Please note that many of these can also contain human voice. If the snippet is from a song with someone actively singing: is that speech or not?
Once you have a crisp definition of speech and non-speech, you can manually create a representative training set that represents accurately your definitions. And then you need to listen to them and see if you can train yourself to distinguish them reliably.
Depending on how exactly you define speech and non-speech, that may be rather easier or pretty much impossible. For example: You can't distinguish "silence in speech" from "other silence".