Picking out a signal that appears to be noise inside a large signal

I have a signal from a photodiode sensor that has two types of noise. One type of noise is ambient light white noise that gets introduced just from the surrounding environemnt. The other type of noise is due to the motion of the sensor.

Currently i don't seem to have a problem filtering out the white noise when the sensor is held still. But when the sensor is moving i get very large noise from motion, this noise is much larger than the signal i am interested in and as a result makes the signal I want look like noise.

The signal i am interested is anywhere between 0.75Hz and 4Hz, the motion signal can be anywhere from 0.25Hz to 5Hz.

I am currently using a butterworth filter between 0.5 and 4 Hz and then a moving average to smooth it out. But this causes the signal to be lost in the motion noise.

I have looked into adaptive filters but am not sure whether i should try to implement them or not because i do not have enough experience with them to tell whether they will help my problem or not.

Are there techniques that are used to pick out signals that share frequency with noise?

• Forgive me for asking a dumb question. How can motion affect the voltage output of a photodiode sensor? By "motion" do you mean the photodiode moves in and out of a beam of light? Sep 22, 2015 at 13:08

Adaptive filters could be used in situations where you have a reference signal, what adaptive filters basically do is to try modelling the signal that is present in captured signal using reference signal. So, we can subtract and/or extract the required signal.

In the situation mentioned in question, you can use adaptive filter if you have reference for either the signal you are interested in or the noise you want to remove. If you have the noise signal as reference adaptive filters will remove the required signal(signal of interest) from input and only noise will be remaining, now this noise can be subtracted from original input signal to get the required signal(signal of interest).

Same could be done if signal of interest is present as a reference, but in this case the adaptive filters output will be the required output.

First, filtering can be touchy. A moving average is kind of destroying the information contained in the signal. It would be of use in situations where the real signal doesn't vary much, such as if you were reading a noisy DC voltage and wanted to have an good idea of the average voltage. I don't want to get into all the details of using a moving average (and I can't pretend I know them all), but since the signal of you photodiode sensor varies quickly, a moving average will smooth it indiscriminately from noise.

Next, you should give at look at the frequency response of your butterworth filter. Again, don't want to get into to much details but when you use a filter, you generally need to use lower and upper frequencies a little bit lower and higher than the min and max ones present in your signal. The reason is that a real (non-ideal) filter won't simply cut everything lower than 0.75Hz and higher than 4Hz and allow the rest to pass undeterred; it starts to attenuate the signal before the cutoff frequency and finishes after. You could check the frequency response on an illustration on the Wikipedia article.

Then, is your signal of interest somewhat of a characteristic signature? I mean, do you have an idea of what it is supposed to look like?

If yes, you could use an auto correlation technique. The idea is to convolve a copy of the known signal to your noisy input signal.

Example: Imagine that you have a (somewhat simplified) sonar and that you emit a chirp signal, which is of a known form. You listen with an antenna for echoes of the chirp who could have bounced on some object. The signal you get form your antenna is pretty noisy and you cannot distinguish the chirp from the noise. Still, you want to detect it because the time elapsed since it's emission (and the speed of sound) allows to calculate the distance at which the object is in regard to the chirp emitter.

A good and simple way to go would be to auto correlate the chirp with the antenna signal. You could think of it as a kind of AND operation, in the sense that as a result you will have a signal where the amplitude would be very low where the chirp is not present and higher where the chirp is present. For more visual people it should look like something like this:

The top "x(n)" shows two "chirps" emitted at some time. (There is no link between them, just an image I generated in the past for an assignment.) In the middle, there is the signal "y(n)" captured by the antenna. At the bottom, you have the output signal "Rxy3", which is the result of the autocorrelation between "x(n)" and "y(n)". As you can see, some time after the emission of the chirp, you can identify the location of the echoes (the peaks).

As for how to implement it, you could check at this example: http://www.mathworks.com/help/signal/examples/measuring-signal-similarities.html

• I know generally what my signal looks like, but as i said it can shift in frequency by a few Hz and also in magnitude. I don't know how specific the signal convolution would require but the motion signal is also sometimes in the same frequency range and changing in magnitude (but consitently a much larger magnitude). What is the green signal in the fist picture? And what is the vertical blue lines in the second? I assume the vertical blue in the first is just zero. Sep 25, 2014 at 21:30
• I changed the picture for an example of mine, should be clearer. Also, I wonder, could it be possible for you to "block" the detector of the photodiode and to sample only the noise caused by the movement? I don't know if it would work but it could allow you to get the characteristics of your noise. If it works, there are methods to filter out the noise if you know what the noise is (imagine noise cancelling headphones) and if it is relevant, we could give a look at this next. Sep 26, 2014 at 13:30
• Yes i believe there are methods to sample where the signal of interest is greatly reduced and noise still present (same AC shape but it would be on a different scale), i do not know if i can fully eliminate the signal of interest though. Sep 26, 2014 at 17:59
• Ok... Well that's all I know without trying to solve the problem myself. I hope someone more knowledgeable will step in and help you! Sep 26, 2014 at 18:03
• Does convolution require that the signals be on the same scale? or just the same shape? or rather, the auto-correlation you used in the example? Sep 26, 2014 at 18:11

If you have the prior information about the desired signal then using Adaptive filters to remove noise of varying characteristics would be the best approach.

Also if you want to use adaptive filters but do not have prior information about the desired signal then see if there are any characteristics in the system which can provide you the prior information about the noise. You can then use a feed forward system to reduce the noise.