ICA for Noise Reduction over a Dataset

Question

Suppose my dataset consists of $N$ example vectors $\mathbf{x}_{1}, \ldots, \mathbf{x}_{N}$ where $\mathbf{x}_{n} \in \mathbb{R}^{p}$ $\forall n$. I assume that each vector $\mathbf{x}_{n}$ is comprised of an underlying true datapoint $\mathbf{s}_{n}$ that is corrupted by additive Gaussian noise $\mathbf{x}_{n} = \mathbf{s}_{n} + \mathbf{w}_{n}$.

What I want to do is estimate the $\mathbf{s}_{n}$. Can I do this with ICA?

---

So the ICA model is of the form:

$$ \mathbf{x}_{n} = a_{1,n} \mathbf{s}_{1} + a_{2,n} \mathbf{s}_{2} + \ldots + a_{N,n} \mathbf{s}_{N} $$

Does this mean I need to fit my problem to the model by assuming that $ \mathbf{s}_{n} = a_{1,n} \mathbf{s}_{1} $ and $\mathbf{w}_{n} = a_{2,n} \mathbf{s}_{2}$ ? If so, how do I recover the vectors $\mathbf{s}_{n}$ $\forall n$?

I ask because I see the ICA problem formulated as:

$$ \mathbf{x} = \mathbf{A} \mathbf{s} $$

Would this then imply that: $\mathbf{s} = \mathbf{A}^{+} \mathbf{x}$, where $\mathbf{A}^{+}$ is the psuedo-inverse of $\mathbf{A}$? But how would I get the $\mathbf{s}_{n}$ out of this formulation?

Note: I cross-posted this on stats.stackexchange to get the statisticians' point of view, but I'd like to hear what the signal people have to say.

---

Edits:

So I do not know $\mathbf{A}$ but I do know that each $\mathbf{s}_{n} \sim f_{s}(\cdot | \theta)$ for some parameter vector $\theta$. I have specified $f_{s}(\cdot | \cdot)$ but $\theta$ is unknown. I also know that the noise vectors are not sampled from that same distribution, but that they are independent of the signal vectors.

Answer 1

This appears similar to a classic least squared solution of an overdetermined equation that proceeds as follows:

Starting with:

$$ \mathbf{x} = \mathbf{A} \mathbf{s} $$

$\mathbf{A}$ is not a square matrix if overdetermined (more equations than unknownns) so therefore an inverse does not exist. What you do then is multiply both sides by the transpose of $\mathbf{A}$ since $\mathbf{A^TA}$ is a square matrix. From that, assuming and inverse exists for$\mathbf{A^TA}$, you can solve for s:

$$ \mathbf{A^Tx} = \mathbf{A^TA} \mathbf{s} $$

$$ \mathbf{s} = (\mathbf{A^TA})^{-1}\mathbf{A^Tx} $$

Answer 2

Your model $\mathbf{x}_n = \mathbf{s}_n + \mathbf{w}_n$ seems too simplistic. It basically says that your output is just some input corrupted by noise. (Unless $\mathbf{s}_n $ is not really your input but some transformed version of it.)

Usually, it's more complicated than that in physical systems. This is why a better model would be $\mathbf{x}= \mathbf{A s} + \mathbf{w}$, i.e., your input $\mathbf{s}$ first undergoes some (noiseless) transformation $\mathbf{A}$, then gets corrupted by noise $\mathbf{w}$. In my opinion, you should start by finding the adequate model to describe your data, then try finding a way to get the unknown quantity of interest to you (which should be represented as an unknown in your model).

Now, if you decide to use the model $\mathbf{x}= \mathbf{A s} + \mathbf{w}$, then the answer of Dan Boschen is what you need for finding $\mathbf{s}$, assuming you know $\mathbf{A}$; technically this is what we call an identification problem; you know the output of a system $\mathbf{x}$ along with the transformation $\mathbf{A}$ that generated it and you want to identify (find) the input $\mathbf{s}$.

If you don't know $\mathbf{A}$, then your problem becomes a blind identification problem where you only know the output $\mathbf{x}$ and what you seek to find this time is both $\mathbf{A}$ and $\mathbf{s}$. This case, is what you can use ICA for.