1
$\begingroup$

I do not have any background in Electrical Engineering and I have recently started a project which involves signals captured from sensors.

Lowpass filtering: The way I understand it is: The values below the CUTOFF are allowed to pass through, above the CUTOFF are simply filtered.

So far so good. Now, how does this work in real world?

The data I have is captured 100 times per second at equidistant intervals, hence my sampling frequency is 100 Hz. This is also understandable.

But now, my CUTOFF is 20 Hz. What does it mean? Because all I have is some digits. None of them is below 20. When I read about filtering it makes me think of CUTOFF as some value from the range of signal itself. I have given multiple explanations to myself.

  1. Somehow, from the signal, find out the frequency of each data point and if that frequency is below 20 Hz, don't do anything. Otherwise put 20 Hz in the output. But I have no idea how to find out frequency of single data point. After all, its just a number, right?

  2. It has to work on multiple values at a time. That is, from one data point to the other, if there is some above average peak, just smooth it out. And this 20 Hz is essentially a limit of how sharp the peak can be.

Please help me understand in the simplest terms. I asked the same question to my guide and he explained me using the terminologies in Electrical Engineering and some more alienate stuff, which was ways and bounds beyond my brain.

$\endgroup$

migrated from stackoverflow.com Jan 3 '17 at 12:07

This question came from our site for professional and enthusiast programmers.

  • $\begingroup$ Start with a "moving average" which is one of the simplest lowpass filters. BTW, your idea #1 is totally wrong, because individual points don't have a frequency. Like you said in #2, it works on multiple values at a time. But unlike your #2, the filter is not trying to decide whether there's a peak to remove, it just smooths everything (at least, if you are talking about linear filters. Detect/trigger filtering methods do exist, but they are very non-linear and hard to reason about). $\endgroup$ – Ben Voigt Dec 31 '16 at 5:43
  • $\begingroup$ Next time consider posting on Electrical Engineering or Cross Validated. $\endgroup$ – Dev-iL Dec 31 '16 at 9:52
  • $\begingroup$ Your explanation 2. is a lot closer to how it works than your 1. But the local shape a filter tries to detect is more like a sinusoid than a peak. $\endgroup$ – hotpaw2 Jan 1 '17 at 9:35
2
$\begingroup$

Any signal can be constructed from the sum of sinusoids of different frequencies (to within an arbitrarily small error). This reconstruction is unique and can be calculated with the Fourier transform.

When we talk about the various frequencies that comprise a signal, we are talking about these sinusoids of various frequencies. In fact, in signal processing, we are used to thinking about the same signal in the time domain -- as a function that gives a magnitude for every instant in time, or in the frequency domain -- as a function that gives a magnitude and phase for every frequency.

When you apply a low-pass filter, you are modifying the frequency domain view of the signal so that all frequencies higherthan the cutoff have their magnitudes set to pretty much zero.

Another way of thinking about it is, you take all the sinusoids that make up the original signal, and throw away the ones with frequency higher than your cutoff.

The secret to how this actually works is called the "convolution theorem": https://en.wikipedia.org/wiki/Convolution_theorem

The convolution theorem shows that convolving two signals together is the same as multiplying their frequency-domain representations together. A "low pass" filter is just a signal that has a constant magnitude for frequencies below the cutoff, and near zero magnitude for frequencies above the cutoff.

When the filter is convolved with the input signal, the input signal's high-frequency components are multiplied by zero, effectively removing them.

$\endgroup$
  • $\begingroup$ "...and throw away the ones with frequency less than your cutoff", you meant throw away the ones with higher frequencies, since we are talking about low pass filter. Pl correct me if I am wrong. $\endgroup$ – user3628070 Jan 2 '17 at 9:10
  • $\begingroup$ @user3628070 - yes. fixed $\endgroup$ – Matt Timmermans Jan 2 '17 at 14:12
  • $\begingroup$ As noted by Matt, when a lowpass filter is applied to a given signal, the signal is changed depending on the properties of the signal and the characteristics of the lowpass filter. The time- and frequency-domain representations of the signal are altered, but how the lowpass filter changes the signal is more readily understood by looking at its frequency-domain representation. $\endgroup$ – Michael_RW Jan 6 '17 at 13:26
0
$\begingroup$

You frequency transform the data using a Fourier transform. Typically you have high values for the low frequencies, and low values for the high frequencies. However discontinuities in the audio will create peaks in the high frequencies.

Essentially you just set the high frequencies to zero and do the inverse transform. However then the sections don't join up. So you use a filter to avoid that problem.

$\endgroup$
0
$\begingroup$

A simple intuitive approach: The simple low pass filter will take the present sample P and add to it a multiplicate portion of previous samples $P_{-n}$.
say sum $\sum w^{i}\cdot P_{-i}, w<1$
This means that you are uncertain about the present sample but know that it's related to previous samples. By diluting the present reading you are suppressing the present noise by adding previous readings in a weighted manner.
Beyond this intuition, you have to start delving into the real theory/interpretation as mentioned in other answers.
Put another way: you are assuming a contamination changes rapidly (sample to sample say) and that the signal (the information you want) is changing much more slowly.
Now the formula above isn't "normalized" so you need to multiplicatively adjust it.

$\endgroup$
0
$\begingroup$

I'll attempt to produce a non-technical simple answer;

Imagine your data plotted on a graph where the horizontal, X axis, is time in 1/100th of a second intervals and the vertical, Y axis position is the number your data represents, e.g. -1000 to 1000. (you have not given the bounds of your data, but this does not matter).

If you draw this graph you will have some sort of wavy line. We call this a time domain plot.

As an example, imagine this line started at 0, then after 10 time intervals went to 1000, then after another 10 time intervals went to -1000, and did so over and over with the shape of a perfect sine wave.

So after 20 time intervals it completed one 'cycle' and this is repeated every further 20 time intervals. Well, we know 1 time interval is 1/100th of a second, so 20 time intervals is 20/100ths of a second. We can find the 'frequency' of this signal by simply dividing 1 second by it. 1s / (20 * 0.01) of a second is 5. These are Hertz (shortened to Hz). The frequency of our signal is 5Hz.

Note that it took more than one point (sample) to define the frequency of a signal. In our case it took 20. Thus it is meaningless to ask the frequency of a single data point in the time domain.

Frequency, for an audio signal, would be it's tone. But equally it could be vibration or temperature or any other physical property that can be measured.

You will just have to accept this next bit; In signal theory, any contiguous signal can be represented at an instant in time as a sum of an infinate series of sine waves, each having just two properties, frequency and amplitude. Amplitude is the Y axis, above. We've just had an example of frequency.

We can also plot this on a graph, but on this new graph the horizontal axis is frequency instead of time. The vertical axis is still amplitude.

Instead of a wobbly line we'd draw a single vertical line from zero to 2000 at the horizontal position of 5Hz.

So imagine the first graph again, but this time it's much more complicated. It wobbles all over the place. It's a complicated mess, but the sample points are still every 1/100th of a second.

According to the previous statement we'd need an infinate number of sine waves to define it, that would be a very big frequency domain plot!

However, here is another statement you will just have to accept; It is that "any sampled data has what is called a Nyquist limit". This states that any sampled signal can only describe fully and accurately the origianl signal with a frequency upto 50% of the sample rate. E.g. Your sample rate is 100Hz, so the 'Nyquist limit' is 50Hz.

So why bother including frequencies above 50Hz in our so called infinate series, the sample rate means we'd never detect them.

Our frequency plot of the more complex signal need have no points above 50Hz.

Why bother having an infinate number of sine waves (even if they stop at 50Hz) if the difference in frequency between them is so tiny that thay are almost the same. We can have far fewer!

Imagine our orignal time domain graph. Imagine that at each sample point we multiplied the sample by some number between zero and one. Obviously if we multiplied all the samples by zero we'd get a flat line. If we used one we'd get the orignal signal back.

But what if we multiplied the points by a different number for each sample, selected from a repeating sequence. We'd get a completly different, yet predictable, result.

But hang on, these point multipliers are themselves a repeating, periodic series of points so they also must have a relationship to frequency domain equivilent. To keep this simple we will just accept that they do.

Lets take this idea further:
So this time we'll create a sequence of, lets call them coefficients, let there be three of them for example; C1=0.1, C2=0.2, C3=0.1.

Starting at our third sample in time (S3), we'll multiple the 3rd sample by the 1st coeficient, the 2nd by the 2nd, 1st sample by the 3rd. We'll then sum the result and make our new 3rd sample that value.

e.g.
S1new = S1
S2new = S2
S3new = (S3 * C0) + (S2new * C1) + (S1new * C2)
S4new = (S4 * C0) + (S3new * C1) + (S2new * C2)
S5new = (S5 * C0) + (S4new * C1) + (S3new * C2)
S6new = (S6 * C0) + (S5new * C1) + (S4new * C2)
....and so on until we've used up all the samples.

This is called convolution.

We could even shift the samples in time down one sample once we'd done all the sums, that way we could go on forever. S2 becomes S1, S3 to S2, etc.

So each Nth sample is modified by the three preceding it and the three coefficients.

This is what we call a digital filter.

There are basicly two sorts of digital filter algorithm, (Finite impluse response) FIR and (infiniate impulse response) IIR. The above example is a FIR. Of these there are three types, low, band and high pass.

The question you might ask is, "How do I know what coefificients to use?"
The complete answer to that is complicated, but there are online tools e.g. Tfilter and special programs (like Matlab, Python, Octave or sMath) that can help you. Just plug in your sample rate, 20Hz cutoff and a few other parameters.

The length of the coeficients is related to the 'order' of the filter, the higher the order the more of the unwanted signal is blocked but the final output takes longer to get as it takes more calculations.

No digital filter is perfect. All have some non-ideal effects on both the in-band (wanted) and out-of-band (unwanted) signal. These effects might be ripple, non-linear gain and a phase shift (a time delay) all of which depend on frequency. Sadly no filter will completely eliminate the out of band signal or have absolutely no effect on the wanted one.

There are filter models that are used to describe these effects with funny names like Chebbyshev, Butterworth and Bessel. I can advise you that Butterworth is a good starting point.

The bottom line: For your data, all you need to decide is the length (order) and what non-ideal shape you require (e.g. Butterworth). Plug this into a program to generate FIR coefficients and implement the code loop listed above to process your data.
Programs have software levers you can play with to allow you to trade off the non-ideal response against the ratio of wanted to unwanted signal.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy