# Denoising a signal using eigendecomposition

I have a complex observable series $$Y(t)$$ which is the result of summing two complex r.v $$X(t)$$ (unobservable) and a $$\epsilon(t)$$ (unobservable).

$$Y(t)=X(t)+\epsilon(t)$$

Assume that $$X$$ and $$\epsilon$$ are independent, and that $$\epsilon$$ is serially uncorrelated and has constant mean and variance.

I construct a Hankel Matrix of the observed series and then I create a sort of variance-covariance matrix by multiplying the later by its conjugate transpose.

$$R_Y=\mathcal{Y}\cdot \mathcal{Y^*}=\mathcal{X}\cdot \mathcal{X^*}+\mathcal{E}\cdot \mathcal{E^*}$$ (The last equality holds because of the independence of $$X$$ and $$\epsilon$$.)

After that, I make the eigendecompositon of that matrix and I obtain:

$$U\cdot (\Lambda_X +\sigma^2\cdot I)\cdot U^T$$ Where $$U$$ is the orthogonal matrix of eigenvectors, $$\Lambda_X$$ is a diagonal matrix containing the eigenvalues (in decreasing order) of $$\mathcal{X}\cdot \mathcal{X^*}$$ and $$\sigma^2$$ represents the "variance" of $$\epsilon$$ (under my assumptions $$\mathcal{E}\cdot \mathcal{E^*}$$ is a diagonal matrix with $$\sigma^2$$ in its diagonal.

Assume that the matrix $$\mathcal{X}\cdot \mathcal{X^*}$$ has not full rank, and because of that it has some null eigenvalues.

By the eigendecomposition of $$R_Y$$ we could select which eigenvalues correspond to the $$\mathcal{X}\cdot \mathcal{X^*}$$ matrix, (by considering the first $$d$$ eigenvalues, where $$d$$ represents the rank of the matrix $$\chi \chi ^*$$) and then we can estimate $$\sigma^2$$.

Knowing $$\sigma^2$$ we could estimate $$R_X=\mathcal{X}\cdot \mathcal{X^*}$$.

Is there any way to factorize $$R_X$$ in order to obtain $$\mathcal{X}$$ and then by the one-to-one relation between Hankel matrices and Series, obtain $$X(t)$$? if this is possible, under which additional conditions?

• What you’re describing is essentially something called Singular Spectrum Analysis (SSA). When you get a chance, look that up on Wikipedia. I think you might find it very helpful. If that is indeed what you’re looking for I can do a full solution showing that yes, under specific conditions you can recover your two signals – matthewjpollard Nov 13 '18 at 16:47
• @matthewjpollard I've been reading about this so call SSA, but I had the feeling that it was slightly different. I'll try to take another look at it. Thanks! – RScrlli Nov 13 '18 at 17:48
• you’ll see it used a lot in reference to de-noising, but you can also use it to extract out separable signals (you just won’t see that application much in literature because it’s more difficult to explain I think). At the very least, the method may be related, curious to hear your thoughts – matthewjpollard Nov 13 '18 at 19:41

So I'll get this answer started by saying again that what you are describing reminds me a lot of Singular Spectrum Analysis. The process is pretty much exactly how you describe: you have some observed signal, $$Y[n]$$, which is some sort of mixture, and we take $$Y[n]$$ and create a Hankel matrix from it using some desired frame length M (this will effectively determine your sub-space resolution).

Now that we have our Hankel matrix, we can either compute $$R_Y$$ directly and then do an eigendecomposition, or we could go with a Singular Value Decomposition of the trajectory matrix. You'll find them to essentially be equivalent if we use the left singular vectors; either method is perfectly valid.

So once we get our matrix of eigenvalues, $$U$$, we need to project them using your typical Karhunen-Loeve method to get what we'll call principal components (this is sort of classic SSA notation, I dislike the naming but I digress):

$$P = Y^H U$$.

Now that we're here, we can use the singular values to determine amplitude of the signals themselves, and the principal components essentially determine the "parts" of the signal, i.e. $$X[n]$$ and $$\epsilon[n]$$.

Let's say we have N principal components, and we know that we only "need" the first one: from our principal components, we perform what is typically called Eigentriple grouping via the following multiplication:

$$C = UP^H$$

This new matrix, C, is what we call the reconstructive component(s) of $$Y$$, and is based on which principal components we selected. We're not quite done yet, becuase $$C$$ is the same dimension as our Hankel matrix, $$Y$$, so do get back to our signal, we'll need an additional step. To reconstruct the series itself, we'll perform what's called diagonal averaging (a bit of a misnomer I think). The goal here is that we'll finally map back to a single signal, let's call it $$Y_out[n]$$. It's in this step that we'll be able to utilize that one-to-one relationship of the Hankel to the series to extract out signals. This step is a bit long winded, so I've omitted it for brevity, but you can consult Wikipedia or any of the many papers/texts on SSA for more information.

So now that we've briefly gone through it, you might find yourself asking "when will this method actually work". SSA, like anything, is a tool and it has its limits. From a signal processing standpoint, it's great at determining individual signals which are fairly narrowband (i.e. they don't cover the entire spectrum) and are "separable". In the context of SSA, "separable" means that the signals are non-overlapping in frequency. For a simple example, let's say I have two sinusoids, one at 50Hz, and another at 500Hz. SSA will be able to easily separate the two signals, and we will be able to isolate the signals pretty easily; we would call these signals separable. However, let's say I change the second signal to be 51Hz instead of 500Hz. In this case, SSA will struggle, as the signals are very, very close to one another; we would call these signals non-separable.

How do we determine which signals we care about? That's the difficulty with this method. If you have some appreciable signal-to-noise ratio (SNR), you can perhaps use that to your advantage and identify which signals are of interest and which signals are just purely noise by inspecting the eigenspectrum; true signals as represented by the principal components will likely be associated with large eigenvalues, whereas noise will be relatively low power, and spread.

So, in short, if you have some time series observation, you can use SSA to attempt to discern individual signals/trends. It's a really powerful tool, and as long as the signals are separable, you should have some reasonable success. Hope that helps!

• Thank you so much! That was very helpful. I'll let you know in the comments if I have success – RScrlli Nov 14 '18 at 7:12