Precision of Computer Vision algorithms

Let's say the task is to determine element position on image. First very important thing is correct detection of object then some algorithms of calculating position are used (for examble blob analysis). Everything depends on multiple things (detection correctness, used algorithms etc.)

Lets assume we have callibrated image and know error given by callibration. What are the methods to calculate reliably precision of computer (and machine) vision algorithms? Can it be done analiticaly or only by experiments and tests?

The question addres cases when we detect element position and also other computer vision problems.

I want to get references to problems which are related to computer/machine vision especially element position detection and present some correctness computations either analyticaly or experimental approach to show this precission.

Also suggestions how to improve this question are welcomed.

For example, Hartley & Zisserman suggest using preconditioning prior to homography estimation, because taking direct matrix inverse can lead to huge errors or instabilities. This applies to any numerical method working with matrix inverse.

Feature detection algorithms often use sub-pixel approximation of interest point location.

Most books discussing numerical methods also deal with their stability analysis.

Sometimes you need to do some statistics to analyze precision and accuracy of your estimator (be it a least-squares estimator or maximum likelihood estimator). This is useful in algorithms like RANSAC, which deal with outliers. You would also like to know, how well the estimated transform fit your data and possibly discard results that are too inaccurate.

When working with finite differencing or doing some filtering, a slight Gaussian blurring is done to remove noise, which would otherwise cause huge errors in second derivatives.

Some problems in computer vision are ill-posed. A method of regularization (such a Tikchonov regularization) is necessary in order to solve them. Examples where this is necessary include computing anisotropic diffusion.

• So this applies when we have detected some features and match them to model features with statistics (and this matching gives error which we can compute). How about computing feature detection errors. For example if features are blobs extracted by thresholding? – krzych Sep 23 '12 at 11:50
• I think you cannot compute "detection error" given only the image. There need to be some context in which you can say the feature is erroneous. – Libor Sep 23 '12 at 12:28
• Exactly but what conntext. How to design some tests to figure out feature detection correctness? – krzych Sep 23 '12 at 12:45
• As H&Z noted in their book: "This is an chicken and egg problem..." We cannot say which features are "good" and which are "bad" without matching them first. There are some developments in designing feature descriptors so that they are matched well to a larger datasets. Given measurement of a descriptor 'quality', you can discriminate features which are not likely to be matched. – Libor Sep 23 '12 at 12:51
• But there must be some method to evaluate correctness of whole system. I think that it is very important for machine vision applications especially when we talk about element positioning. As I said in question I am also interested in some ways of testing this correctness. – krzych Sep 24 '12 at 20:43