Understanding Matched Filter, delay

Question

I am trying to build a filter for the detection of signals with the known form in noisy and low-amplitude data.

I used a simple example in Python provided here: https://www.youtube.com/watch?v=XXLV9g6RQQ4

but for the data I have.

initial_signal is a numpy array, it has length of 8000 points.

(for each element of initial_signal array we have the x-value - coordinate E for the signal.) it's important to note, that the distance between sample points can vary, so

E[i] - E[i-1] != E[i+1] - E[i]

(the x-grid is nonuniform)

FIR_coeffs = numpy array it has length about 57 points.

It was obtained by reversing of theoretically known signal - to match.

fir_coeffs = template[::-1]

det = signal.lfilter(fir_coeffs, 1, init_signal, zi=None)

But, the result actually is always shifted because of the delay of the filter - see the figure.

Is it possible to shift back the result of filtration - to "remove the delay" between the signal and filtration result?

In general - I need to get zero-phase matched filtrated signal as a result to compare it with the initial signal and calculate the coordinates of regions of interest using filtration results.

Maybe there is a better way to do matched filtering using Python in this case? Please provide me with examples if possible. Thanks in advance.

Answer 1

Any causal filter will have delay since a causal filter is the weighted sum of past inputs (and for an IIR case past outputs as well).

If the filter is a linear phase filter (as guaranteed by using an FIR filter with symmetric coefficients), the delay is conveniently $(N-1)/2$ where $N$ is the number of coefficients in the filter. Given that, if you use an odd length filter (N odd), there will be an integer sample delay and in post processing you can simply remove the first $(N-1)/2$ samples from the output to guarantee a zero-phase result. It is feasible to to do resampling for fractional delay correction, but that is not nearly as straightforward and has opportunity to introduce more distortion if not done properly.

The other possible option is to use a zero phase filter, applicable to post processing and limited to the cases when the matched filter can be implemented with symmetric coefficients (linear phase). This is implemented with the filtfilt command in Python's scipy.signal (as well as Matlab and Octave). This runs the signal through the filter twice (both in the forward and reverse direction), resulting in the zero-phase result, so the coefficients for the filter used would need to be related to the required "matched filter" coefficients as follows:

$$h[n] = h_1[n] \ast h_1^*[-n]$$

That is since the filtfilt command runs the signal through the filter in the forward direction, followed by running that through the reverse filter (coefficients in reverse order and complex conjugate if the coefficients are complex), then in order to achieve the same result as a standard matched filter with coefficients $h[n]$, the relationship above must apply. What we also see from this relationship is that $h[n]$ in this case will be forced to be symmetric.

Given this complexity, unless the factored coefficients were readily available to implement $h_1[n]$ above and $h[n]$ is conveniently symmetric, I would not advocate this approach and rather simply remove the leading samples from the output consistent with the expected delay.

If the filter is not a linear phase filter, correcting for expected delay can be more complicated, since group delay is the negative derivative of phase with respect to frequency: what this means is the delay for different frequency components in the signal will be different. If we were to equalize this to correct for the delay variation, we would basically be converting the filter into an equivalent linear phase filter. In many cases with a known waveform pulse shape, it is easy enough to determine the delay for that given pulse and then subtract that from the result to get relative delay of all pulses.

I demonstrate this below with a non-linear phase exponential pulse as follows:

I created a test waveform with a known delay for two pulses, one starting at 600 samples, and the other other at 1000 samples:

The output of the matched filter without further processing is given below.

Evaluating this result, the processing delay of the filter is 399 samples (given the pulse used has most of its energy at the start of the pulse, the reverse used in the matched filter will have most of its energy at the end: a maximum phase filter; and the resulting delay is nearly the length of the filter).

So in this case, it is simple enough just to remove the first 399 samples to compensate for the processing delay of the filter:

Zooming in we see the results consistent with the 600 sample and 1000 sample delay: