Let's say I'm doing feature extraction from some sensor reading using a DTW algorithm. I want to extract and catalogue a set of features.
Before I send my signal over to the DTW extractor, I have several tools to remove what I consider to be noise from the signal. I can use bandpass/notch filters or DFTs.
When doing this interactively, it's easy for me as a human to spot which part of the spectrum is interesting and which part of it is low entropy noise.
has three signals superimposed.
- a low frequency square pulse (400 samples long - 2.5mHz)
- a mid range sine wave (33 samples long - 30mHz)
- and a high frequency noise.
To me, this is trivially visible. I say visible, but this is also audible if I were to listen to the signal.
The ground noise happens to be the RF interference that a brushless fan makes when it sporadically comes on. So I could have the noise there or not there depending a completely random variable.
It's not clear whether the middle frequency sine wave is simply a superposition on the square wave, or if it is being generated by the same fundamental underlying phenomenon. If it is not the same phenomenon, there's a possibility that their respective phases will drift, and the rising edge of the square signal will only sometimes align with the rising edge of the sine wave. This means that for what ends up being a square wave signal, I could have dozens and dozens of different looking DTW features, and this is something I'd rather avoid.
Having described the above scenario, my questions are as follows:
- what metric can I use (or even theory or branch of analysis if the answer isn't a simple one) to determine the amount of filtering necessary? For any given analysis window of say 500 samples, how do I determine programatically how to preprocess my signal before trying extract features?
- given a sample window with a particular spectral distribution, does an optimal cutoff for a bandpass filter exist? (or a notch frequency)
- if so, does it always exist?
- if an optimal value exists, how do I obtain it?
- what is the relationship between this and the amount of information in the signal?
Notes: the vagueness of my question is part of my problem. I don't know where to start looking for answers.
To be clear: I'm not attempting to get more information than there is. I'm wanting to degrade my signal as little as possible while removing obvious low entropy noise.
I've looked into things like adaptive filters, but these seem to often time have a separate measurement to aid in the feedback loop.