# What kind of filter I should use to remove the oscillations in this autocorrelation function?

I have an autocorrelation function which is shown as following,

I do believe these trailing oscillating wigs are spurious and should be removed by some kind of filter. But I am not familiar with any types of filters. Any help will be appreciated! The original data can be downloaded at

~

Update @ Apr 14 2017:

This result was processed with the filter suggested by Dan. • What would be the meaning of a filtered autocorrelation? The autocorrelation of a function tells you something about the function's properties; if you filter it, what would you learn about the function? – MBaz Apr 20 '17 at 1:52
• @MBaz Thank you very much for your prompt response! As we see in the autocorrelation function, there is a fixed frequency oscillation which does not damp out as t increases. Could you tell me what I could learn from this autocorrelation function then? – Pandaaaaaaa Apr 20 '17 at 2:35
• The peaks with frequency of (roughly) 2Hz (i.e. two peaks per second) tell you, that your original signal has periodic contents of period 0.5s. – Maximilian Matthé Apr 20 '17 at 6:19
• @MaximilianMatthé Thank you very much for your reply! As this autocorrelation function computed from a time series data, is it possible to remove this 2 Hz oscillation when calculating the autocorrelation function? – Pandaaaaaaa Apr 20 '17 at 12:45

The following comb filter structure can be used to filter your data prior to computing the autocorrelation which will significantly reduce or eliminate the ringing due to the repetition in your signal. However be aware that it may also be filtering signal content based on the spectral occupation of your signal. The structure of the signal and the frequency response is shown below (for N=101). Where N is the number of samples to be a 0.5 second delay. Implemented in Matlab using

out = filter([1 ones(1,N-1) -1],1,in) This filter above as given is a highpass response so depending on the spectral content in the signal of interest, this may not be ideal.

Another option that will equally create nulls at the repetition rate and harmonics of the repetition rate is a moving average filter: Implemented in Matlab using

out = filter([ones(1,N)],N,in)


Note in this case the filter was normalized, a 2 could be used as the second parameter in the first case to normalize that filter as well. Note that the first filter removes DC and is a highpass function which may not be desirable, while the second filter has a 1/f frequency roll-off as a sinc function in addition to removing the harmonic nulls which will likely widen the autocorrelation response. If the high frequency content is desired, a third filter option that would be interesting to see can be done based on this post

Transfer function of second order notch filter

This filter also removes DC but can have a much tighter nulling response and flat elsewhere based on a nulling parameter $\alpha$

By inserting zeros in the frequency response we can get the base spectrum to repeat, so in your case you would insert N-1 zeros to perform the following filter function (see referenced post for structure of this filter):

out = filter((1+alpha)/2*[1 zeros(1,N-1) -1], [1 zeros(1,N-1) -alpha], in)


The frequency response for $N = 10$ and $\alpha = 0.9$ is shown below. (Your N of course is much larger). If you vary $\alpha$ less than or more than what I used up to but not equal to 1 (I recommend using $\alpha$ in the range between 0.7 and 0.999); the nulls get sharper as $\alpha$ approaches 1. If you do implement these on your data, please post the results for all these cases so we can see the difference it has caused.

• Thanks a lot for your input! I just done an analytical calculation and found that 0.5s oscillations are not interested in my problem. They should be removed before compute the autocorrelation function. I am not familiar with this type of filter, could you provide more about how should I implement this algorithm? Where I can get a Maltab code or python code. Thanks again! – Pandaaaaaaa Apr 20 '17 at 14:35
• @Pandaaaaaaa what is the number of samples in your dataset to implement a 0.5 second delay? The filter is quite simple: out = filter([1 0 0 0 0 0 0 0 ... 0 0 0 0 0 -1],1, in) Where the +1 and the -1 are seperated by enough 0's to create the 0.5 second delay. – Dan Boschen Apr 20 '17 at 14:38
• The number of sample in the time series data is 45,000 and sampled at time difference 0.004. The "filter" you mention here is a Matlab function? – Pandaaaaaaa Apr 20 '17 at 14:42
• Yes. Just type help filter in matlab to learn more about it. Also look at the zeros() command as a way to make a long string of zeros. Good luck! But if I calculated right you want 125 sample delay so I think it would be out = filter([1 zeroes(1,124) -1],1,in). I could be off by a sample in my delay but let us know how it works! This filter is available in python too, I believe as lfilter as part of signal. – Dan Boschen Apr 20 '17 at 14:43
• Thank you Dan, I have implemented this algorithm! I posted the result in the question please take look at it! I feel like it works, however, not quite perfectly. As the autocorrelation function has been tremendously changed as t is close to zero. – Pandaaaaaaa Apr 20 '17 at 15:07