Why do we need to estimate eigenvalues?

Question

I am not working in signal processing field, but recently I happen to read a paper which estimates source numbers using Gerschgorin radii, and I feel kind of confused about why we need to estimate eigenvalues (this might be a silly question in your point of view). I google it and some answers say the reason is that it is hard to get eigenvalues in some cases. But I still feel confused. It seems to me that we can directly calculate some or even all the eigenvalues even though the matrix is in relatively huge sizes, for example, $1000\times 1000$, with the formulation $AA^T x=\lambda x$, which can be done in a few second in Matlab. Is there anything wrong with the way I think?

Answer 1

It seems to me that we can directly calculate ... which can be done in a few second in matlab.

Who says that Matlab is calculating it directly, or that it isn't using Gershgorin's circle theorem in its algorithm? Can you look at the code to see what they're doing?

Finding the eigenvalues of an $n \times n$ matrix is tantamount to factoring an $n^{th}$-degree polynomial. Factoring any polynomial of degree greater than $n = 4$ is an iterative process, which works better if you can start with a good guess at the roots.

So, when Matlab does an eigenvalue decomposition of your $1000 \times 1000$ matrix, it is not "directly calculating" it -- it's guessing at values, then refining guesses, then reducing the matrix, etc., etc.

Pursuant to that, Gershogorin's circle theorem looks like a handy way to not only do get a set of initial guesses, but to bound the set of possible eigenvalues in a manner that's easy to test for if your algorithm starts to stray.

Is there anything wrong with the way I think?

I would say that believing that typing something into Matlab and getting an answer is the same thing as directly calculating it is an extreme error in thinking. Someone knows how to solve the problem - you hope. But if all you know how to do is press buttons in Matlab and get an answer, the set of persons who really know how to solve the problem doesn't include you.

Answer 2

We do use eigenvalues, because they behave like invariants in linear systems (invariant inputs are well connected to outputs) and generally finding invariants of unknown or model systems provides a great deal of information on how they act on other inputs.

Most problems solved using methods based on eigenvalues rely on some technical hypotheses, related to the properties of signals and noises. They are often modeled with statistical models (stationarity, second-order properties) from which one can derive quantities like the number of sources, the power of the noise...

Now, from the actual observations you have recorded, you will have to compute estimators of the theoretical covariance matrix, and this will be affected by biases, errors and model mismatch. As a results, the eigenvalues are only estimates as well.

Even if there exist some algorithms to solve part of the problem (diagonalizing a matrix) with arbitrary precision in a numerical analysis sense, this does not entail that you have exactly identified the actual signal processing system you were studying.

Then, as answered by @TimWescott, also come all the computational issues.