If I understand correctly, should I consider the hidden states as some kind of representation for words, phonemes, and/or letters?
Go with features, relevant for the task. One of the points of doing this with ML is that you don't need to come up with human categories and mark your signal according to these, which we call feature engineering; the machine learning algorithm is designed to find these itself. And the mechanism by which it does this is only optimized for prediction that are "good" by some metric, not for finding "words", "phonemes" or "letters", which are categories that it doesn't know or care about.
The result might, but usually does not, correspond to "classical" properties of speech.
My dataset consists of short sentences with varying lengths, and I want my model to be able to generate predictions of different lengths. I'm wondering what the most appropriate approach would be and would like to understand the reasoning behind it: Should I pad all samples with zeros to make them the same length, then extract MFCC features to have observations of uniform length (with the hope that the model would learn to associate the zero-filled parts as empty words)? Alternatively, should I train the model with observations of varying lengths?
This really depends. "Predicitions of different lengths" is a pretty mighty topic. Generally, I don't know what hmmlearn offers, but for time-signal analysis (i.e., audio), there's architectures that are "fed" with the signal in a streaming manner; this often suggests recurrent structures. Compare, for example, WaveRNN:
So, there's no general answer to this: you might make a network with a gigantic input layer that can take the maximum audio length as input, and simply prepend zeros (that sounds more sensible than appending them, as you want to predict what comes after, not before).
But it's more likely that you'll feed one sample vector ("sample vector" might mean a fixed-length MFCC vector, or raw PCM samples) after the other until you've processed the whole snippet and "primed" the network for prediction.
