# Uncovering data from noise

I've recently started experimenting with DSP, specifically, understanding digital signals that are generated by some small applications I am creating. For this quest, I've decided to use The Scientist and Engineer's Guide to Digital Signal Processing. While the provided material is beyond exceptional, the more I read the more confused I become as to how I could apply different forms of DSP in my "computer based" experiments.

Here is what I am currently working on:

I have a capacitative sensor. When the sensor is stretched, the capacitance increases. Contrary is true when it contracts. Image below displays this characteristic: An interesting observation I noticed is that when I stretch the sensor, while bending it, my signal is buried in noise. This is evident between 800 and 1000 samples in the above photo. Because I am experiencing this issue, I want to uncover the stretch/contraction of the sensor from this noise; hence my goal of studying DSP.

An interesting comment made by the author, is that when ever we deal with Digital Data we will always want to analyze our Time Domains using DFT. The most efficient algorithm to do this today is FFT. I've also concluded ( high level ) that using the mentioned algorithm is ideal as it will give me a sense of what "sin waves" generate the "digital wave" I am presented with. Hence, I should be able to parse out "sin waves" of interest to re-construct missing points of interest hidden in noise.

Now, while I understand that the change to the frequency domain can tell me that "a spike has occurred", what I am really interested in is re-constructing my wave to be less noisy. Hence, I'd like to derive out a consistent wave form between 800 - 1000, alike I have between 400 - 600.

Questions

1. Since my understanding is still fairly low, is DFT and FFT really the solution to my problem?

2. Would it be logical to take the FFT , filter out outliers that are generated and cause noise, and re-construct the time domain signal back ( with the results that were acceptable ) using iFFT? That should, in theory, give me back a time domain with less noise, no?

3. Is there a better section of DSP that I should read that can help me analyze the decouple noise from interesting data in a time domain?

• Welcome to SE.DSP! I think instead of 800 and 1000 Hz you mean 800 and 1000 *samples* – Peter K. May 22 '17 at 17:11
• @PeterK. ah, you are correct. :) fixed. – angryip May 22 '17 at 17:19

Since my understanding is still fairly low, is DFT and FFT really the solution to my problem?

All the FFT (or DFT) really gives you is a way of visualizing the data in the frequency domain.

Your data may be good for this, except that there will be a large spike at "DC" (the zero frequency bin) because your data has a non-zero mean.

Would it be logical to take the FFT, filter out outliers that are generated and cause noise, and re-construct the time domain signal back ( with the results that were acceptable ) using iFFT? That should, in theory, give me back a time domain with less noise, no?

If you know the approximate frequency of the variation you're after, yes, you can use the FFT (and IFFT) to filter out frequencies different from the band where the frequency will be.

One simple thing you could try without using an FFT is to use a DC blocker. That will make your signal approximately zero mean, and will also smooth out some of the roughness you see.

Is there a better section of DSP that I should read that can help me analyze the decouple noise from interesting data in a time domain?

One place to start might be chapter 15 of that book.

I suspect you'll want to design a bandpass filter around the frequencies of interest and filter out everything except this band. You will still see some effect of the noise, but it should be greatly reduced.

If that isn't good enough, then you may need to try more advanced techniques which don't appear to be covered by the book.

• wrt to the second comment, using fft and ifft. Given the fft output, if I remove frequencies that are not of interest, will the exclusion of those points create gaps in my wave over time? Or should they just make the ifft output more smooth ( more like samples from 400 - 600, and less like samples from 800 to 1000 ) ? – angryip May 22 '17 at 18:43
• To do that properly, you'll need to design an FIR filter. If you have $N$ data points and your filter is length $M$ then the FFT length needs to be $N+M-1$. Just zeroing out parts of the spectrum means your effective filter length is $M=N$... which will result in "time aliasing", which is probably not what you want. Try it, but don't expect it to be in any way accurate. – Peter K. May 22 '17 at 18:50
• @Peter K: Not to broaden to scope too much here, but can the time aliasing you mention be reduced or removed by zero-padding the N data points so that the input vector to the FFT is longer than N? – Michael_RW May 23 '17 at 18:00
• @Michael_RW : Yes, of course: provided the FFT length is at last $N+M-1$ long, the time aliasing will be reduced. The trouble is, zeroing out FFT bins means you're forcing $M$ (the filter length) to be the same as the FFT length... which means you always get aliasing. It means you need to decide your filter length $M$ in the time domain, take its FFT of length $N+M-1$ so you can then multiply it by the FFT of the signal, and then take the inverse FFT. – Peter K. May 23 '17 at 18:04