0
$\begingroup$

I have some process, which produces some rectangular signal. I have an algorithm, which recovers this signal. To test my algorithm I compare the recovered signal with known: enter image description here

My question is:

Could You help me with some common and well-established methods to describe the quality of my detector?

Of course, I am able to calculate the number of non-detected jumps, delay its deviation. The question is that I don't want to use homemade solutions. Hope there should be a big science with well-established methods and criteria to perform such tests.

$\endgroup$
2
  • $\begingroup$ If the pulses are carrying bits, why not use the bit error rate? $\endgroup$
    – MBaz
    Mar 19, 2018 at 16:46
  • $\begingroup$ They are not bits. I mean it is not just on/of signal it has duration. In any case I will read about bit error rate. $\endgroup$
    – zlon
    Mar 19, 2018 at 16:49

2 Answers 2

1
$\begingroup$

It is a rare individual who develops detection algorithms as a recreational pursuit. If you are not such a person then the reason for your algorithm should provide, what is favorable and not favorable in some measurable way.

Your question has some ambiguity because random delay, missed transitions, and false transitions is all you have mentioned. "Big Science Tests" are based on probabilistic assumptions (a model) that applies. If your transitions were random with some rate $\mu$, a Poisson model would be promising. One might look at why you don't see a one when there is a one which is the error mechanism, which typically in Signal Processing is associated with noise.

Detection is usually cast as a Hypothesis Test with alternative hypothesis $\mathbf{H_0}$ and $\mathbf{H_1}$ where one hypothesis is signal present and the other is signal absent.

Another sort of hypothesis can be tied to a parameter. As an example, A Poisson Process has a mean rate $\mu$ which can be $\mathbf{H_0}$ and $\mathbf{H_1}$ can be the case the observed mean rate $\mu_{observe} \; \ne \;\mu$

As Mbaz suggests in his comment, missed and false detections can be combined as an error probability which might look like

$$P_{error}= P( \text{ones received}| \text{zeros sent}) P(\text{zeros sent})+ P( \text{zeros received}| \text{ones sent}) P(\text{ones sent}) $$ if the ones and zeros are independent.

The essentail point is that you need to understand your data model to use a standard test.

This is not an exhaustive list of types of statistical tests you can use. and many in signal processing have intractable forms. There are paired tests. There are non parametric tests. There are robust tests. Statisticians are inventing new tests every day.

Your data model on the other hand, is up to you.

Lastly, making things up is actually common and as long as it isn't fake, is often acceptable.

$\endgroup$
0
$\begingroup$

Feels like a standard cross-correlation would do a good job.

$\endgroup$
2
  • $\begingroup$ But cross-correlation did not include false detected and not detected, hope there are more advanced methods. $\endgroup$
    – zlon
    Mar 19, 2018 at 16:34
  • $\begingroup$ that's not true. For any misdetection, the duration of the misdetection will directly be subtracted from the crosscorrelation coefficient. $\endgroup$ Mar 19, 2018 at 21:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.