0
$\begingroup$

So, In my report I am comparing the related works, In most of the previous work, researchers have used spectrogram as input to vanilla CNN and classify. Where I am using different handcrafted features from the signal and then feeding it to DNN.

I want to explain why the second method works better than first, for that I am comparing like this

Motivated by prior work, Several AI-based approaches have been proposed to classify sounds using signal analysis. Most of them use short-term magnitude spectrograms transformed from sound data as input to Convolutional neural network (CNN). The drawback of this method is, Since CNN is based only on a spectrogram input, some domain-specific important characteristics might get missed in the feature space. Such as Zero crossing rate(ZCR), Skewness, Shannon Entropy, Fundamental frequency (F0), Formant frequencies etc

I just wanted to know, Is this statement correct from DSP aspect? because I am saying in my statement that spectrogram doesn't contain zero-crossing rate(ZCR), Skewness, Shannon Entropy, Fundamental frequency (F0), Formant frequencies etc

$\endgroup$
3
  • $\begingroup$ The statement is correct because of the wording "might get missed". Indeed, the statement does not state that spectrogram do not contain the aforementioned information but rather the information is changed, and may also be reduced in information theoretic sense (this is true for every data processing) en.wikipedia.org/wiki/Data_processing_inequality $\endgroup$
    – AlexTP
    Oct 1 '20 at 8:45
  • $\begingroup$ @AlexTP But Spectrogram already contains the Fundamental frequency (F0), Formant frequencies? $\endgroup$ Oct 1 '20 at 9:27
  • $\begingroup$ It depends. Even if it is done properly and no information loss, "might get missed" and "will be lost" are different. $\endgroup$
    – AlexTP
    Oct 1 '20 at 12:26
3
$\begingroup$

It's almost a matter of philosophy, i.e., difficult to argue hard facts.

On the one hand all the features you mention can be extracted from the raw signals. So in theory the network should be able to learn how to do that if they provide meaningful information for the task at hand. This is what part of the ML community is claiming: feature engineering is dead, let the networks figure out themselves what are the best features, we're only biasing them by what we think are good features, which may be wrong.

On the other hand, if you do have very good reasons to believe that some features are particularly useful, it can actually help the network. It wouldn't need to learn to discover such features first and thus, it could train much faster. This is related to the concept of known operator learning, where you embed fixed operators (that you know in advance should be beneficial to your task) into the network to make it train faster. I'm leaning towards this approach since it often makes sense.

If people used complex-valued STFTs to the input, no data were lost and all the features you mentioned (ZCR, F0, ...) could still be extracted from it. In a spectrogram, the phase is discarded, so one can argue that in fact some information is lost. If you just use your features as input though, it still does not represent the entire signal so some information is still lost and you may have to argue why you think this information is irrelevant.

Here is where it is difficult to argue hard facts and the only thing you may be able to do is show actual training results to make your point.

$\endgroup$
2
  • $\begingroup$ 1) Feature engineering is still critical depending on task complexity and dataset size; 2) nothing 'philosophical' about this, statistical at best with confidence bounds $\endgroup$ Oct 2 '20 at 5:25
  • $\begingroup$ Okay, "philosophical" is maybe the wrong term. But otherwise I'm all with you - I do agree that feature engineering is important (just saying that part oft he community claims it's not anymore) and you can analyze it statistically which I meant by saying study actual training results (statistically). $\endgroup$
    – Florian
    Oct 2 '20 at 8:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.