A book reports that ICA cannot be used if the independent components of the analyzed data are Gaussian (at most one can be Gaussian, but no other). However, in the same book, the following example is reported:

A number of 1024 samples of a two-dimensional normal distribution was generated with mean μ and covariance matrix Σ. Similarly, 1024 samples from a second normal pdf were generated with the same covariance matrix and mean −μ. For the ICA, the method based on the second- and fourth-order cumulants, presented in this section, was used. The resulting transformation matrix W [...]

The example continues using the ICA showing the independent component found. However, my question is: how was it possible to use ICA if the data comes from two normal/Gaussian distributions (and not just one)?

The complete example is available here at pages 283,284.

The book is Konstantinos Koutroumbas, Sergios Theodoridis - Pattern Recognition.


2 Answers 2


The example given in the book Konstantinos Koutroumbas, Sergios Theodoridis - Pattern Recognition (4th Estition):

enter image description here

The point in this example is to show the property of the method used: Find a projection direction where the distribution of the data looks the least Gaussian.

Pay attention that the data given to the algorithm isn't a linear combination of the 2 variables but having both of them as the distribution which means the data itself is clearly not Gaussian.

Indeed, the algorithm resulted in a direction which a projection along it will have a distribution which is Bi - Modal, clearly very far from being Gaussian, which is the property of the solution the algorithm seeks.


I don't know the rest of the problem statement, nor the result they arrive at, but consider this:

If you add two independent normal random variables, you just get a new normal variable (with its mean being the sum of the original means, and the variance being the sum of the original variances).

This derives from the convolution property of pdfs; if you you convolve two Gaussians, your convolution integral just happens to give you another instance of a Gaussian.

This also extends to the twodimensional case.

So, you can never tell apart the sum of two independent sources from one source, and there's uncountably many different ways of arriving at the same sum. Since the density is the same, so are all moments, and cumulants.

However, the problem statement mentions a fourth-order cumulant method, which I'm not familiar with. And it also does not claim the first and second normal normal random variables to be independent – the trick here might be that the result shows it can detect at least the presence of a Gaussian mixture here. In fact, the problem statement doesn't exactly state these were added together; I could also imagine, just based on the text excerpt, that it's 2048 samples, the first half of which have mean µ, and the second half mean of –µ.

  • $\begingroup$ thank you for your answer! the complete example is available here at pages 283,284. $\endgroup$
    – volperossa
    Commented Oct 18, 2022 at 10:50
  • $\begingroup$ I am not sure about your reasoning. More accurately, it is from this property of the Gaussian Distribution but not the reasoning. For the Gaussian case 2nd moment data is all you can do, namely do decorrelation of the data. This is done perfectly with PCA hence no reason to do it with ICA (Which, in optimal case, should match PCA for the Gaussian Case). $\endgroup$
    – Royi
    Commented Oct 20, 2022 at 19:03
  • $\begingroup$ not fully following there, @Royi – you add up two independent Gaussian-distributed RVs, you get a new Gaussian-distributed RV; all the moments will be that of a Gaussian of the sum variance. There's no degree of freedom there. $\endgroup$ Commented Oct 20, 2022 at 19:04
  • 1
    $\begingroup$ Indeed the ICA problem for the Multi Gaussian case is ill poised and the reasoning is as you mentioned. Yet usually in this context it is more justified by the other side of the same coin, For Gaussian data all we can do is decorrelation which is the PCA. This is the optimal (The ICA optimal) case for Gaussian. $\endgroup$
    – Royi
    Commented Oct 20, 2022 at 19:10
  • 1
    $\begingroup$ ahhh together with your answer this all makes sense! $\endgroup$ Commented Oct 20, 2022 at 19:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.