# How to calculate the Diagonal loading factor evaluate calculate the inversion of a covariance matrix

I am programming a Generalised Likelihood Ratio Test (GLRT) detector. When it comes to inverting a covariance matrix $Ri$, I need to do a diagonal loading to fix the problem of sigularity of this matrix, but I don't have any info about the technique or algorithm that allows to estimate the right diagonal loading factor factor $\sigma$.

Rather than calculate inverse ($Ri$) I have to calculate inverse ($Ri +\sigma \cdot I$) where $Ri$ is the covariance matrix.

Short answer: just use $\sigma = 10^{-8}$.
Covariance matrices have eigenvalues $\geq 0$ (theoretically),
so $Ri + 10^{-8} \, I$ will have eigenvalues $\geq 10^{-8}$, safely non-singular.

split Covar = S + N, "signal" + "noise", by eigenvalues or by SVD, Singular-value decomposition aka PCA, Principal component analysis. This has several advantages:

• gives you an idea of how noisy your Covariance is -- a good thing to know
• you can choose a cutoff, e.g. 98 % of the sum of the eigenvalues, or a number of terms, e.g. 3, to plot and look at
• fast, rock-solid implementations: in Python, numpy eigvalsh, numpy svd and sparse scipy svds (sparse, dense or LinOp).

Be careful with large covariance matrices: eigenvalues can be < 0 because of roundoff, and they can be very noisy -- see here on stats.stack .

See Press et al. for a good discussion of SVD: "Solution by use of Singular-value decomposition",
Numerical Recipes p. 793 ff.

• thanks for all, I did give the value 1e-9 and 1e-8 to sigma , and it did correct the problem of sigularity. About the PCA, the great thing that I am trying to understand it nowadays for an other issue I have ; maybe it will fix both issues . thank you again for the response and the link. – Linda bendjama Mar 1 '18 at 16:56