# How do I quantify the amount of information being lost by filtering a digital signal?

Let's say I'm doing feature extraction from some sensor reading using a DTW algorithm. I want to extract and catalogue a set of features.

Before I send my signal over to the DTW extractor, I have several tools to remove what I consider to be noise from the signal. I can use bandpass/notch filters or DFTs.

When doing this interactively, it's easy for me as a human to spot which part of the spectrum is interesting and which part of it is low entropy noise.

For instance, the following signal:

has three signals superimposed.

• a low frequency square pulse (400 samples long - 2.5mHz)
• a mid range sine wave (33 samples long - 30mHz)
• and a high frequency noise.

To me, this is trivially visible. I say visible, but this is also audible if I were to listen to the signal.

The ground noise happens to be the RF interference that a brushless fan makes when it sporadically comes on. So I could have the noise there or not there depending a completely random variable.

It's not clear whether the middle frequency sine wave is simply a superposition on the square wave, or if it is being generated by the same fundamental underlying phenomenon. If it is not the same phenomenon, there's a possibility that their respective phases will drift, and the rising edge of the square signal will only sometimes align with the rising edge of the sine wave. This means that for what ends up being a square wave signal, I could have dozens and dozens of different looking DTW features, and this is something I'd rather avoid.

Having described the above scenario, my questions are as follows:

• what metric can I use (or even theory or branch of analysis if the answer isn't a simple one) to determine the amount of filtering necessary? For any given analysis window of say 500 samples, how do I determine programatically how to preprocess my signal before trying extract features?
• given a sample window with a particular spectral distribution, does an optimal cutoff for a bandpass filter exist? (or a notch frequency)
• if so, does it always exist?
• if an optimal value exists, how do I obtain it?
• what is the relationship between this and the amount of information in the signal?

Notes: the vagueness of my question is part of my problem. I don't know where to start looking for answers.

To be clear: I'm not attempting to get more information than there is. I'm wanting to degrade my signal as little as possible while removing obvious low entropy noise.

I've looked into things like adaptive filters, but these seem to often time have a separate measurement to aid in the feedback loop.

• You need to define what you consider as "amount of information". If you talk about mutual information, then any reversible operation (e.g. non-ideal low-pass) does not change the amount of information in the signal. Also, noting the Information Processing Inequality (en.wikipedia.org/wiki/Data_processing_inequality), no operation can increase the amount of mutual information in a signal. Commented Nov 28, 2016 at 9:48
• Can you please edit the question for clarity? It would be ideal if you could add a little bit more information about your application. For example, what is this "Ground" noise? Is it actual geological noise that finds its way in the measurements? Is it a baseline background noise? Is it noise because a reference or grounding electrode is misbehaving? Is it something else? It is impossible to answer the specified questions, especially about "information", without knowing a little bit more about the signals.
– A_A
Commented Nov 28, 2016 at 10:00
• A_A: I've done my best to clarify. I feel it's not an ideal question because it doesn't necessarily have a concise answer, and maybe there's no generalized answer to what I'm looking for. Maximilian: I'm most definitely not trying to extract information which isn't there. Commented Nov 28, 2016 at 13:42

• what metric can I use (or even theory or branch of analysis if the answer isn't a simple one) to determine the amount of filtering necessary?

You can use what you already have, that is, the "strengths" of the components that result from the analysis. You can sort the coefficients in decreasing order and then obtain their cumulative sum. Finally, you can set a threshold and stop adding components once the cumulative strength of the chosen components goes (for example) above 90%. In this way you would be selecting the $N_{sc}$ strongest components and rejecting components that have higher signal-to-noise ratios (or in other words are closer to some background noise level). If this has to be a low pass filter, then after you have isolated the first $N_{sc}$ components that contribute to 90% (for example) of the total signal strength then you can set the low pass filter cut off frequency at the component with the highest frequency (or bin).

• For any given analysis window of say 500 samples, how do I determine programatically how to preprocess my signal before trying extract features?

Please see previous answer, given the frequency (or scale) limits imposed by the length of the window (here 500 samples), you might have to adjust thresholds and cut offs. Please note that if you try to apply the above algorithm per frame (of 500 samples each), then, inevitably, you will get some sort of modulation between frames which, depending on your application you will have to manage somehow. This is because the signal strength will be changing between windows of 500 samples, which means that the potential cut off of the low pass filter will be changing as well, which means that each frame will bear different amounts of filtering. The easiest way to deal with this would be to increase the window overlap (at the expense of more frames and therefore more calculations).

• Given a sample window with a particular spectral distribution, does an optimal cutoff for a bandpass filter exist? (or a notch frequency)

Only under specific assumptions about the spectral distribution of the noise. In the absence of this, you can study your signal and try to create a Matched Filter that is tailored to the spectral characteristics of the signal and therefore anything outside that would be filtered out.

• if so, does it always exist?