# How to obtain the probability of a detection?

I will use a specific example from image processing to illustrate my question, but I'm actually interested in the higher level / abstract procedure. I must lack the specific vocabulary and my hunch is that my question relates to some topic in math / signal processing / computer science that I'm not yet aware of.

I extracted part of an image and I want to check if it contains an edge. Using some filtering technique (e.g. Sobel) I obtain pixels which are more likely to be part of an edge.

What I would like to obtain is the likelihood of the image to contain an edge. For example: The probability that this image contains an edge is p=0.82 .

Q1: Which processing steps am I missing here?

My intuitive approach is the following: Let's assume I want to purchase a magazine. I know that the average price is 4€. By checking how much money I carry, I can predict the likelihood that I can complete the purchase. By example:

• With 1€, the likelihood is low, maybe 0.1
• With 4€, the likelihood is average, 0.5
• With 50€, I can afford almost any magazine, 0.99

Now, I drew the probabilities and created a PDF (probability density function) out of nowhere. Knowing the actual repartition of magazine prices would help me create a more realistic PDF.

Q2: Is this intuitive approach correct?

Q3: How does it apply to the aforementioned problem in image processing?

Q4: Which topic/keywords is my question actually about? Can you recommend some sources / references to develop my knowledge?

You have a fairly common problem in Detection theory. Let's assume you have a $N \times N$ grid of pixel values and $N$ is small enough that the edge is a straight line in the region. For that given line, you can formulate a hypothesis on whether and edge was on that line or not. You might also have to some other information like the amplitude of the edge and distribution of the non edge pixels. Unfortunately, you usually don't know where the lines are, so one thing people do is something called a Generalized Likelihood Ratio GLR test. So, if you can enumerate all the possible lines that can fall in your $N \times N$ grid and test each, you can detect it, assuming you can estimate the other parameters. Unfortunately, that's a big search space and probably not a good approach. Even if you could do the search, since pixels will be contained in more than one search, the tests will not be independent and a closed form solution for a Detection probability will be difficult if not intractable to calculate. As an alternative, you can do an $M$ out N test, test if there are $n$ out of $N \times N$ pixels that are part of a line. We don't know $n$ either but if there is a single line $n$ is bound by $\sqrt{2} N$. Again we have nonindependent tests and a difficult calculation, and the pixels that are high aren't necessarily connected as in a line.
A more practical approach is to choose $H_0$ to have no edges, and assuming we have a reasonable sampling distribution, we would look for small $p$ values. The original likelihood we mentioned would be more powerful and often people use the ideal case as an upper bound.
Radon transforms are good at finding lines, so that is a possible approach as well. Picking $N$ essentially a heuristic as well.