3
$\begingroup$

I have a digital on/off signal sampled at a regular interval. There was a problem with the sensor that produced the signal which has sometimes caused a regular on/off ticking when the sensor should be in the off state. I.e. sometimes when the readings should be zero, there is a square wave instead (but with an unequal mark:space).

How would I go about designing a filter to remove this square wave (i.e. output a zero if there is this regular ticking occurring, output the input value otherwise)?

The ticking has a frequency of around 16 minutes, the events that the sensor was observing might last anything from one minute to a few hours. I expect that this means that perfect recovery of the correct data is impossible, but I’d like to get as close as possible.

I would think some kind of notch filter is what I need, but everything I’ve found so far seems to be about filtering analogue signals.

(My memory of signal processing courses is a bit patchy, hopefully I’ve used the right terms here).

EDIT:

Thanks for your comments so far. I've attached an image illustrating what I am trying to achieve. Also I wrote the wrong frequency for the regular ticking - it's actually 16 minutes, not 8, but I doubt that affects the overall approach.

@Jan - yes, ideally I would just have a better sensor, but this is the data I have for now, I don't currently have the option of switching to a different sensor.

EDIT 2:

The data processing is happening offline, so non causal solutions are fine.

Sample data

$\endgroup$
  • 2
    $\begingroup$ Could you provide a plot to visualize the problem, please? Also, are you asking the correct question? If this sensor behaves so badly, why not use a different one? $\endgroup$ – jan Oct 17 '13 at 12:01
  • $\begingroup$ Without a graph of the data, it's hard to say, but I'd guess some kind of non-linear filter like a median filter would help? $\endgroup$ – endolith Oct 18 '13 at 21:50
  • 1
    $\begingroup$ So for a correctly-detected event, the sensor should be in the on state for between 1 minute and a few hours? That seems to mean that the spurious "ticks" don't really differentiate themselves from what could be correct events. I don't think you're going to be able to do better than ad hoc heuristics that can suppress some of the false alarms. It's probably up to you to determine what is best for your application. $\endgroup$ – Jason R Oct 22 '13 at 1:09
  • $\begingroup$ @JasonR that's right, the only way that the ticks differentiate themselves is that when they happen, they happen regularly at that fixed 16 minute frequency. What's actually happening is that this is a wireless sensor that transmits a heartbeat signal every 16 minutes to the base station, unfortunately for some of the sensors I have installed, the heart beat transmission triggers the sensor. $\endgroup$ – Andy S Oct 22 '13 at 5:29
  • 1
    $\begingroup$ In the example graph, all the "bad" ticks are shorter than all the "good" ticks. Is this always true? If so, can you simply filter on the tick length (not frequency)? $\endgroup$ – mtrw Oct 22 '13 at 13:45
2
$\begingroup$

Are you processing this data offline or in real time? If you're doing offline processing, or can live with a significant delay, then you could apply a simple filter as follows:

  • From each input sample, check to see if the sensor reported "on".

  • If it did, search for a "few" 16-minute periods in each direction and see whether the sensor reported "on" there as well. This is a noncausal operation; if you're processing data in real time, you can search in both directions by first applying a large enough delay to the signal.

  • If there are "enough" consecutive "on" samples at the spurious period in either direction, suppress the "on" indication from the sensor.

This should suppress the unwanted square wave of "on" indications, but as indicated in the comments, it also could possibly suppress valid "on" reports. The "few" and "enough" parameters you'll have to select yourself based upon what is reasonable given the overall behavior of your system. Basically, you want to set "few" so that you're looking at enough spurious periods to make a good decision. You'll want to set "enough" small enough to suppress short bursts of the spurious period, but large enough to avoid suppressing too many valid "on" indications that happen to be separated by 16 minutes.

$\endgroup$
  • $\begingroup$ The data processing is happening offline, so this is definitely an option (will update question to make that clear) $\endgroup$ – Andy S Oct 22 '13 at 23:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.