I am programming a Generalised Likelihood Ratio Test (GLRT) detector. When it comes to inverting a covariance matrix $Ri$, I need to do a diagonal loading to fix the problem of sigularity of this matrix, but I don't have any info about the technique or algorithm that allows to estimate the right diagonal loading factor factor $\sigma$.

Rather than calculate inverse ($Ri$) I have to calculate inverse ($Ri +\sigma \cdot I$) where $Ri$ is the covariance matrix.

Need help, please


Short answer: just use $\sigma = 10^{-8}$.
Covariance matrices have eigenvalues $\geq 0$ (theoretically),
so $Ri + 10^{-8} \, I$ will have eigenvalues $\geq 10^{-8}$, safely non-singular.

A longer answer:
split Covar = S + N, "signal" + "noise", by eigenvalues or by SVD, Singular-value decomposition aka PCA, Principal component analysis. This has several advantages:

  • gives you an idea of how noisy your Covariance is -- a good thing to know
  • you can choose a cutoff, e.g. 98 % of the sum of the eigenvalues, or a number of terms, e.g. 3, to plot and look at
  • fast, rock-solid implementations: in Python, numpy eigvalsh, numpy svd and sparse scipy svds (sparse, dense or LinOp).

Be careful with large covariance matrices: eigenvalues can be < 0 because of roundoff, and they can be very noisy -- see here on stats.stack .

See Press et al. for a good discussion of SVD: "Solution by use of Singular-value decomposition",
Numerical Recipes p. 793 ff.

  • $\begingroup$ thanks for all, I did give the value 1e-9 and 1e-8 to sigma , and it did correct the problem of sigularity. About the PCA, the great thing that I am trying to understand it nowadays for an other issue I have ; maybe it will fix both issues . thank you again for the response and the link. $\endgroup$ Mar 1 '18 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.