I am working on Voice Disorder Detection problem. I have extracted 13 MFCC features, 13 delta and 13 delta-delta features from each audio file (2 to 4 secs). I extracted these features for each frame (consisting of 23 ms). So for example an audio divided into 75 frames has 75 examples in my dataset i.e each frame act as individual example in the dataset.
I trained Deep Neural Network on this dataset (70% of dataset) and evaluated my model on the rest 30% of the dataset giving me accuracy of 96% and 84% respectively.
But the problem is, while testing my model on a single unseen audio data when I calculate MFCCs frame by frame and feed every frame to the model, the model prediction for every frame is not the same.
So my question is:
What should I do so that my model will give consistent prediction for all frames belonging to the same audio file?
I tried taking the majority of the class predicted by my model for that audio but that too gave me incorrect result.
One thing that I think I might have done wrong is that I haven't used every frame belonging to a particular audio file for training and so some of them are in test dataset. Is that a problem or am I missing something important here?