8
$\begingroup$

Let's say the task is to determine element position on image. First very important thing is correct detection of object then some algorithms of calculating position are used (for examble blob analysis). Everything depends on multiple things (detection correctness, used algorithms etc.)

Lets assume we have callibrated image and know error given by callibration. What are the methods to calculate reliably precision of computer (and machine) vision algorithms? Can it be done analiticaly or only by experiments and tests?

The question addres cases when we detect element position and also other computer vision problems.

I want to get references to problems which are related to computer/machine vision especially element position detection and present some correctness computations either analyticaly or experimental approach to show this precission.

Also suggestions how to improve this question are welcomed.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
5
$\begingroup$

For example, Hartley & Zisserman suggest using preconditioning prior to homography estimation, because taking direct matrix inverse can lead to huge errors or instabilities. This applies to any numerical method working with matrix inverse.

Feature detection algorithms often use sub-pixel approximation of interest point location.

Most books discussing numerical methods also deal with their stability analysis.

Sometimes you need to do some statistics to analyze precision and accuracy of your estimator (be it a least-squares estimator or maximum likelihood estimator). This is useful in algorithms like RANSAC, which deal with outliers. You would also like to know, how well the estimated transform fit your data and possibly discard results that are too inaccurate.

When working with finite differencing or doing some filtering, a slight Gaussian blurring is done to remove noise, which would otherwise cause huge errors in second derivatives.

Some problems in computer vision are ill-posed. A method of regularization (such a Tikchonov regularization) is necessary in order to solve them. Examples where this is necessary include computing anisotropic diffusion.

Improve this answer
$\endgroup$
12
  • $\begingroup$ So this applies when we have detected some features and match them to model features with statistics (and this matching gives error which we can compute). How about computing feature detection errors. For example if features are blobs extracted by thresholding? $\endgroup$
    – krzych
    Sep 23, 2012 at 11:50
  • $\begingroup$ I think you cannot compute "detection error" given only the image. There need to be some context in which you can say the feature is erroneous. $\endgroup$
    – Libor
    Sep 23, 2012 at 12:28
  • $\begingroup$ Exactly but what conntext. How to design some tests to figure out feature detection correctness? $\endgroup$
    – krzych
    Sep 23, 2012 at 12:45
  • 2
    $\begingroup$ As H&Z noted in their book: "This is an chicken and egg problem..." We cannot say which features are "good" and which are "bad" without matching them first. There are some developments in designing feature descriptors so that they are matched well to a larger datasets. Given measurement of a descriptor 'quality', you can discriminate features which are not likely to be matched. $\endgroup$
    – Libor
    Sep 23, 2012 at 12:51
  • $\begingroup$ But there must be some method to evaluate correctness of whole system. I think that it is very important for machine vision applications especially when we talk about element positioning. As I said in question I am also interested in some ways of testing this correctness. $\endgroup$
    – krzych
    Sep 24, 2012 at 20:43
4
+75
$\begingroup$

This doesn't answer the whole question, but it addresses part of what the OP asks about.

It can only be done experimentally. To do it analytically would require information about what the algorithm should have returned. But to know that, you need a known always correct computer vision algorithm to compare against (as well as detailed analytical descriptions of the image being tested against). Analytical solutions require knowledge of a ground truth which is analytical rather than hand generated on a case by case basis. But we don't have an analytical way to generate a ground truth -- that is what we are trying to develop.

Given that it can only be done experimentally, you may want to look at google scholar. If you are after people location there will be a lot of papers dedicated to locating a person, or parts of a person like a head or hands. Car location will also have a lot of speciallized attention. Other objects will just want generic algorithms.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Some references could improve this answer. $\endgroup$
    – krzych
    Oct 1, 2012 at 16:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.