0
$\begingroup$

First, I have reviewed close-match questions, but can't find something that seems to match what I'm looking for. Also, it's been 30 years since college, so forgive me for forgetting stuff.. :)

I am analyzing IP Packet data. I am creating bins (of varying window sizes.. The window seems not to be sufficient to remove/smooth the harmonics) to count the frequency of certain attributes/behaviors within the data. From this, I am trying to determine fundamental frequencies that activities are occurring. I believe that the artifacts that I am seeing are harmonics since the data tends to be pulse/bursty, resulting in what amounts to a square wave. (see below) . In this sample data, as you can see, there is a strong signal occurring every hour, over the course of 47 hours, yet it is almost impossible to "see" because of the harmonics. (at least, that's my guess/intuition)

Frequency, DFT, Power Spectrum Density

Is there something that I can do to smooth things out so that the harmonics are suppressed (or largely suppressed) and I only see the fundamental frequencies? I feel as though I've forgotten something basic.

$\endgroup$
  • 1
    $\begingroup$ You mean something like a low pass filter? $\endgroup$ – fibonatic Jul 19 '19 at 19:40
  • $\begingroup$ @fibonatic Maybe, but I'm not sure. That will remove the higher harmonics, but since I have no idea what the frequency will be, how can I arbitrarily choose to apply a filter? Also, can't I end up with negative frequencies as a result of mapping the complex data in this way? In the data I have pictured, I think I'm using a 60 second window, but the frequency turns out to be at the once per hour mark. $\endgroup$ – David Hoelzer Jul 19 '19 at 21:03
  • $\begingroup$ Are you just looking for the fundamental period of the signal? Or are you trying to gather some other information from the signal? $\endgroup$ – Tendero Jul 20 '19 at 15:35
  • $\begingroup$ Just the fundamental frequencies/periods of strongly coherent signals in the data. $\endgroup$ – David Hoelzer Jul 20 '19 at 16:44
2
$\begingroup$

If you are interested in finding the periods, have a look at autocorrelation lag. Maybe try some xcorr options too. MATLAB example code:

nh = 47;  % 47 hours.
% Pulse period of 1 hr and make sequence length not a multiple of period
n = 3600*nh + 273; % 1 sec sampling rate
x = zeros(1,n);
% Create a pulse train with varying heights and some noise
x(777:3600:end) = 8 + 2*rand(1,nh);
x = x + rand(1,n);
% Look at lags up to 4 hours
xcorr(x, 4*3600, 'unbiased'); 

Zero lag is in the middle of the figure. enter image description here

Note that the consistent heights show reasonably consistent correlation at integer multiples of the base period. If you want to tolerate timing jitter in the period, you can filter the data with a lowpass FIR filter first, which will smear the peaks in the data and, of course, smear the peaks in the autocorrelation., e.g.,

b = hamming(15);  % Smooth, simple, positive, symmetric, integer delay
x = filter(b,1,x);
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.