8
$\begingroup$

1. Original Situation

I have an original signal as a column data matrix n channels data x:mxn (single), with m=120019 the numer of samples and n=15 the number of channels.

Also, i have the filtered signal as a filtered column data matrix x:mxn (single).

The original data is mainly random, centered at zero, from sensor pickups.

Under MATLAB, am using save with no options, butter as highpass filter, and single for casting after filtering.

save essentially apply a GZIP level-3 compression over a binary HDF5 format, hence we could assume the filesize is a good estimator of the information contents, i.e. maximum for a random signal, and close to zero for a constant signal.

  • Saving the original signal creates a 2MB file,

  • Saving the filtered signal creates a 5MB file (?!).

2. Question

How it is possible the filtered signal has a bigger size, considering the filtered signal has less information, removed by the filter?

3. Simple Example

A simple example:

n=120019; m=15;t=(0:n-1)'; 
x=single(randn(n,m));
[b,a]=butter(2,10/200,'high'); 
 xf=filter(b,a,x);
save('x','x'); save('xf','xf');

creates 6MB files, both for for the original and filtered signal, which is bigger than the previous values due to using pure random data.

In a sense, indicating that the filtered signal is more random than the filtered signal (?!).

4. Evaluative Example

Consider the following:

  • A filter created from a random signal $x_r$ from gaussian noise $\sim N(0,1)$, and a constant signal $x_c$ equal to $1$.
  • Disregard the data type, i.e. let's use only double,
  • Disregard the data sizes, i.e. let's use one column data vector of 1MB, $n=125000$, $m=1$.
  • Lets consider the $a$ parameter as the Randomness Index for testing: $x=\alpha x_r+(1-\alpha)x_c$, meaning $\alpha=1$ is fully random and $\alpha=0$ fully constant.
  • Consider a highpass butterworth filter with $w_n=0.5$.

The following code:

%% Data
n=125000;m=1;
t=(0:n-1)';
[hb,ha]=butter(2,0.5,'high');
d=100;
a=logspace(-6,0,d);
xr=randn(n,m);xc=ones(n,m);
b=zeros(d,2);
for i=1:d
    x=a(i)*xr+(1-a(i))*xc;
    xf=filter(hb,ha,x);
    save('x1.mat','x'); save('x2.mat','xf');
    b1=dir('x1.mat'); b2=dir('x2.mat');
    b(i,1)=b1.bytes/1024;
    b(i,2)=b2.bytes/1024;
    i
end
%% Plot
semilogx(a,b);
title('Data Size for Filtered Signals');
legend({'original','filtered'},'location','southeast');
xlabel('Random Index \alpha');
ylabel('FIle Size [kB]');
grid on;

With the following chart as result: enter image description here

This simulation reproduces the condition of the filtered signal always having a notorious bigger size than the original signal, which contradicts the fact that a filtered signal has less information, removed by the filter.

$\endgroup$
  • 6
    $\begingroup$ I think your question is more about the compression algorithm than anything else. Save the two files with the -nocompression option then go check the bit patterns you are unwittingly generating. My guess is that you random signal actually contains significant repetitions which compress well, while the filtered version does not. Interesting nonetheless :) $\endgroup$ – zeFrenchy Aug 12 '17 at 8:32
  • 1
    $\begingroup$ With no compression, all signals have the same size, 1MB, as the length and data types are all the same. I will check though. I am blindly assuming the compression works as information, so i will put an additional evaluative example twist for checking this "information" aspect... $\endgroup$ – Brethlosze Aug 12 '17 at 12:47
4
$\begingroup$

+1 on very interesting and insightful experiment.

Some thoughts:

  1. It's not true that filtered signal has less information. It depends on your input signal, filter type, and cut-off frequency.
    When you high-pass the noisy signal, you're removing the slowly changing components. That makes your signal composed of 'more frequently changing random numbers', thus more random. Of course, that depends if your input signal contains high frequencies or not. Your input is noise, so contains every high frequency. But if your input is a more ordered signal, it will lose much of its energy after a certain HP cut-off frequency, the output becomes near zero, less random, less size. I think if you increase the cut-off frequency of your HP filter pretty high, after a certain point, the file size will decrease.
    One other experiment would be to pass the signal through a LP filter with a low cutoff frequency and see the difference.

  2. Based on the same theory in 1., you're high-passing your signal, essentially removing the DC part, xc, and leaving it with noise xr.

$\endgroup$
  • 2
    $\begingroup$ Information-theoretically, your 1. is at least half-wrong. The filtered signal must contain less (or at most, the same) information as the unfiltered. $\endgroup$ – Marcus Müller Aug 12 '17 at 9:22
  • 2
    $\begingroup$ @MarcusMüller I definetely agree with you on this (obvious statement) but I have the following concern: had you interchanged the roles of the filter's impulse response and the input random signal (i.e., the (deterministic) impulse response becomes the input to the filter with a random impulse response now) could we still say that the information at the output is less than the information at the input ? $\endgroup$ – Fat32 Aug 12 '17 at 10:09
  • 1
    $\begingroup$ @Fat32 that is an interesting angle! True point. In this particular case I'd argue that if we consider the LPF as the information-bearing signal, then we'd find it contains very little info at all (being very correlated, by design, and rather short). $\endgroup$ – Marcus Müller Aug 12 '17 at 10:19
  • 1
    $\begingroup$ @Fat32 this is a nice suggestion. If i compress the FT of the signal, for a lossless case, i should still have the same size! (disregarding the fact that some parts of the spectrum could lead to less valuable, easy to discard, information). If not, we would have discovered a better compression algorithm, which i honestly doubt :). So i will prepare a second evaluatice example with this approach. $\endgroup$ – Brethlosze Aug 12 '17 at 13:02
  • 1
    $\begingroup$ @MarcusMüller It can't be generalized that simply. You should define things to make that statement true. Assume you have a system that randomizes its input. When input is DC, it makes it random, So your output has more entropy.(It's not difficult to imagine that, just a communication channel that adds noise to the input would do this.) For LTI systems, we know how they treat noise, so that's a different topic. Information theory doesn't base its results on whether a system is LTI. Still I am not an expert to be sure, but I think that is not true. $\endgroup$ – doubleE Aug 13 '17 at 3:06
2
$\begingroup$

I would check 2 things:

  1. If the filter applied is Low Pass Filter or a different filter. If it is a filter which amplifies the noise, the result is reasonable.
    It seems that you use butter() in a form which generates High Pass Filter. Since the input signal is composed of noise, the High Pass Filter amplify it and causes to less compressible file. For instance, try [hb, ha] = butter(2, 0.5, 'low'); where it should support better compression of data (Suppression of noise). If you want to go even farther use [hb, ha] = butter(2, 0.1, 'low');.
  2. Verify that the output of the filter command is single as well. I think that since your filter is double the output is double hence the size of the signal is multiplied. In your code, replace xf = filter(hb, ha, x); with xf = single(filter(hb, ha, x));. What are the results now?
$\endgroup$
  • 1
    $\begingroup$ @hyprfrcb, Try the same with butter(2, 0.5, 'low');. What happens then? $\endgroup$ – Royi Aug 12 '17 at 13:19
  • 1
    $\begingroup$ So the problem is solved. You're using High Pass filter which amplifies the noise hence you got larger file as noise is less compressible. Enjoy... $\endgroup$ – Royi Aug 12 '17 at 13:25
  • 1
    $\begingroup$ You can try [hb, ha] = butter(2, 0.1, 'low'); to see the file size gets even smaller. $\endgroup$ – Royi Aug 12 '17 at 13:27
  • 1
    $\begingroup$ High Pass applied on Noise usually deteriorate the SNR in a signal. This is what you did above. The High Pass filter amplified the noise energy which means the data is less compressible. $\endgroup$ – Royi Aug 12 '17 at 13:42
  • 1
    $\begingroup$ Random signal which is filtered by an LPF filter creates correlation between its samples, hence it is compressible to an higher level. It doesn't work with HPF which operates on random signal. $\endgroup$ – Royi Aug 12 '17 at 14:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.