A Machine Learning Based Algorithm as an Alternative to the Matched Filter

Question

Consider we have to detect a known signal added with Gaussian noise. In this scenario, the matched filter is known to be an optimal filter for SNR. The question: is there any machine learning algorithm/tool that can be used instead of matched filter? Can we have any other detection technique instead of filter (convolution)?

Answer 1

The idea is to have a simple experiment to see if we can get, for a known signal, a better results than the Matched Filter for time delay estimation.

Experiment Objective

Generate, using ML (DL), a system which estimates time delay under the model of a shifted signal with Additive Gaussian White Noise (AWGN).

Experiment Motivation

It is known that the performance Matched Filter for time delay estimation is far from the CRLB at Low SNR scenario. It is not known if the CRLB is too optimistic (As it isn't guaranteed to be a tight lower bound) or the Matched Filter (MLE for this case) is not optimal for such case. This could be some kind of an empirical way to assess which one of the alternatives is more probable.

Optimality of the Matched Filter

Pay attention that the Matched Filter is the global optimum for detection by being the operation, linear or non linear, which maximizes the SNR (Easy to prove with Cauchy Schwartz). Yet this optimality is in the context of detection (There is or no signal of interest). This experiment is about estimating the delay of the signal.

Data

I created a simple Gaussian signal:

The signal is built with 201 samples.

Then I created 1,000,000 signals with shifted versions of the signal with various levels of added noise (The noise STD was in the range of [0, 1]).

For example, here is a shifted and noisy signal:

In he above, the signal was shifted by 3 samples. In order to make things easier I kept the shift to be an integer. It was done to prevent the need to create interpolation phase, etc...
The support of the shifted signal was 300 samples. Hence we had 100 different possible shifts (1 to 100).

I created 2 data sets: Train (1,000,000 signals) and Test (100,000 signals).

I created the simplest 1D CNN using Keras:

modelNet = keras.Sequential()
modelNet.add(keras.layers.Conv1D(32, vSignal.shape[0], activation = 'relu'))
modelNet.add(keras.layers.Conv1D(48, 51, activation = 'relu'))
modelNet.add(keras.layers.Conv1D(64, 25, activation = 'relu'))
modelNet.add(keras.layers.Conv1D(64, 13, activation = 'relu'))
modelNet.add(keras.layers.Flatten())
modelNet.add(keras.layers.Dense(units = 100, activation = 'softmax'))

The above is classification network, basically having to choose between 100 classes (Each per shift).

Trained it for 21 Epochs and got accuracy of 30.05% (Some checkpoint got even better):

Now, what would the Matched Filter do?

It gets 29.52%. So the 1D CNN got 0.5% more accuracy. Not bad for ~60 minutes work.

Let's have a look on the per signal performance of all:

In the above we see the best check point of the net training.

Zooming in to a specific delay (38):

The interesting thing is that the Matched Filter doesn't miss at high SNR (The left part). Since the overall of the 1D CNN is better it means it mainly beats it in lower SNR.

This validates my assertion, one could beat the Matched Filter for time delay estimation in the Low SNR zone and a great candidate to do so is using CNN.
By the way, even in the RMSE the Matched Filter was beaten by the 1D CNN (2.6954 vs 2.7771).

Few remarks:

• Pay attention that the 1D CNN didn't get any information about the signal itself (The reference). Result would be better if it did. One way to do it, the simplest, is to let it work on the cross correlation data. In my opinion well tweaked model working on the cross correlation can get over 35%.
• The experiment should be done for a regression model with fractional delay. Though it is valid for cases the quantization holds (For instance if the sampling frequency is high enough).

Answer 2

Sure, you can learn the matched filter, as convolution with a filter is just a function applied to a signal, and e.g. Neural Networks (through the universal approximation theorem) are good function approximators.

But the fact that the matched filter is optimal in AWGN channels means exactly that: it's optimal, you can't do better than it. So there's really no point, other than showing you can learn it, in learning it. The classical, analytical methods work, and their provably the best.

Interesting problems arise when you e.g. want to minimize error in cases the channel is not AWGN, your reception is irregularly quantized etc., i.e., in cases where the analytical derivation of an optimal receive processing does not exist (or only exists in high complexity).