ICA and Gaussianity: A Misleading Example in the Book Konstantinos Koutroumbas, Sergios Theodoridis - Pattern Recognition

Question

A book reports that ICA cannot be used if the independent components of the analyzed data are Gaussian (at most one can be Gaussian, but no other). However, in the same book, the following example is reported:

A number of 1024 samples of a two-dimensional normal distribution was generated with mean μ and covariance matrix Σ. Similarly, 1024 samples from a second normal pdf were generated with the same covariance matrix and mean −μ. For the ICA, the method based on the second- and fourth-order cumulants, presented in this section, was used. The resulting transformation matrix W [...]

The example continues using the ICA showing the independent component found. However, my question is: how was it possible to use ICA if the data comes from two normal/Gaussian distributions (and not just one)?

The complete example is available here at pages 283,284.

The book is Konstantinos Koutroumbas, Sergios Theodoridis - Pattern Recognition.

Answer 1

The example given in the book Konstantinos Koutroumbas, Sergios Theodoridis - Pattern Recognition (4th Estition):

The point in this example is to show the property of the method used: Find a projection direction where the distribution of the data looks the least Gaussian.

Pay attention that the data given to the algorithm isn't a linear combination of the 2 variables but having both of them as the distribution which means the data itself is clearly not Gaussian.

Indeed, the algorithm resulted in a direction which a projection along it will have a distribution which is Bi - Modal, clearly very far from being Gaussian, which is the property of the solution the algorithm seeks.

Answer 2

I don't know the rest of the problem statement, nor the result they arrive at, but consider this:

If you add two independent normal random variables, you just get a new normal variable (with its mean being the sum of the original means, and the variance being the sum of the original variances).

This derives from the convolution property of pdfs; if you you convolve two Gaussians, your convolution integral just happens to give you another instance of a Gaussian.

This also extends to the twodimensional case.

So, you can never tell apart the sum of two independent sources from one source, and there's uncountably many different ways of arriving at the same sum. Since the density is the same, so are all moments, and cumulants.

However, the problem statement mentions a fourth-order cumulant method, which I'm not familiar with. And it also does not claim the first and second normal normal random variables to be independent – the trick here might be that the result shows it can detect at least the presence of a Gaussian mixture here. In fact, the problem statement doesn't exactly state these were added together; I could also imagine, just based on the text excerpt, that it's 2048 samples, the first half of which have mean µ, and the second half mean of –µ.