1
$\begingroup$

Through this platform, I want to ask that how can I remove unwanted noise from the signal when you do not have much information regarding the frequency at which they appear? Data is collected from an inductive sensor and sampling frequency is 30000 Hz. There are a lot of electrical noises in the signal, the easily distinguishible have already been removed using a Notch Filter. However, there are some electrical noises which are not easily distinguishible from the other parts of the signal. I tested the following approaches:

  • Removing easily visible electrical noises using Notch Filter
  • Taking FFT and applying binning to visualize other noisy parts
  • Using find_peaks() function to detect and remove noisy parts

However, when I appled ifft I could not get the filtered original time series data.

This is the Original Time Series Signal:

Original Time Series Signal

This is the complex Fourier Transform before binning and find_peaks():

enter image description here

And this is the fourier transform after binning:

enter image description here

And this is the result obtained after find_peaks():

enter image description here

And this is the inverse FFT result that I have obtained after so called processing (FFT, binnig, find_peaks() technique)

enter image description here

Can anyone please help me understand where I went wrong?

This is the reference data obtained usinf Eddy Current Sensor. I suppose my data should also look likr this after denoising.

enter image description here

$\endgroup$
8
  • $\begingroup$ Hi Aisha, welcome to SP.SE! We're going to need a lot more information here. What are the characteristics of your signal? Give as much detail as possible (what kind of sensor: microphone, accelerometer, etc; type of data: audio, seismic, etc; sampling rate; frequency spread: narrow-band, broadband, etc). What are the characteristics of your noise? You need to share as much info as you can, and time / frequency plots to see what we're dealing with here. $\endgroup$
    – Jdip
    Oct 19, 2022 at 18:29
  • $\begingroup$ Sure, lemme edit my post. $\endgroup$ Oct 19, 2022 at 18:31
  • $\begingroup$ Filtering depends on knowing what you want to keep and what you want to reject (think of a coffee filter -- little stuff, like molecules, is tasty and desirable; big stuff is gritty and bitter). Can you describe what you expect your desired signal to be? As a start, filtering to only pass the desired stuff and filter out all the rest may work. $\endgroup$
    – TimWescott
    Oct 19, 2022 at 20:03
  • $\begingroup$ While we're at it, you say it's an inductive sensor -- what are you sensing with it? Are you sensing something like magnetic fields (in which case you want what comes out of the sensor as-is), or is it a proximity sensor or an LVDT, in which case the information you want is riding on some sort of modulated signal. $\endgroup$
    – TimWescott
    Oct 19, 2022 at 20:03
  • $\begingroup$ Yes, this is an inductive sensor and it is creating signals on the basis of the magnetic fields generated. And honestly, I don’t know how exactly the signal should look. I just observed some electrical spikes and tried to remove them. $\endgroup$ Oct 19, 2022 at 20:48

1 Answer 1

0
$\begingroup$

The train moves at speed $v_t$. This (absolute) speed reasonably will have a supremum $v_{t\max}$. It obviously has an infimum of $0$, but there is nothing to measure with a train standing around. So we declare some minimum speed we are interested in and call that $v_{t\min}$.

The rail fasteners are spaced at a distance of $d$, which can be assumed as constant for these purposes. The transitions at the "heart" points can with some reason be assumed to be in the same order of magnitude as $d$.

From this, we can derive the part of the spectrum we are interested in. For the lower corner frequency, we take $v_{t\min}$ and add some margin, say, a decade, below. That gives us $$f_l = 0.1\frac{v_{t\min}}{d}$$ We do the same for the upper corner frequency and set the margin again to a decade, above this time of course: $$f_u = 10\frac{v_{t\max}}{d}$$ With these two corner frequencies, design a bandpass to filter the signal. You should be able to see a lot more than before. From there further steps can be taken.

Remark on your attempts so far: The operations you performed on the spectrum where nonlinear, so the inverse FFT was bound to produce rubbish.

$\endgroup$
11
  • $\begingroup$ The distance between the rail fastners can be known. But what if I do not have information regarding the speed of train owing to the data provided? Secondly, you are saying that my approach on filtering was incorrect? $\endgroup$ Oct 20, 2022 at 9:30
  • $\begingroup$ If the speed of the train is unknown, make some reasonable assumptions. It most probably won't go faster than 200mph. Secondly, I did not say it was incorrect. It was just nonlinear (at least the find_peaks function). That makes transforming the signal back into the time domain pointless. $\endgroup$
    – Max
    Oct 20, 2022 at 9:56
  • $\begingroup$ The approach that you have mentioned is certainly interesting. However, I am thinking to go for a lazy approach: I will apply notch filters on each frequency till 100 Hz (becase either ways we are going to ignore samples greater than 100 Hz). I am going for this lazy approach because I have a deadline to follow for the submission of my Thesis $\endgroup$ Oct 20, 2022 at 10:15
  • $\begingroup$ I have also tried using 'noisereduce' tool of python as mentioned in this link: pypi.org/project/noisereduce/…. Can this be helpful? $\endgroup$ Oct 20, 2022 at 10:18
  • $\begingroup$ Just to clarify more: what exactly do you want to do? Above, you said you want to "classify rail switch". Could you elaborate? $\endgroup$
    – Max
    Oct 20, 2022 at 10:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.