Given random varaible X with distribution

$$ \begin{cases} \mathbb{P}\left(X=1\right)=\alpha\\ \mathbb{P}\left(X=-1\right)=1-\alpha \end{cases} $$

Where $ \alpha $ is a given parameter($X$ is a binary signal), and noise $Z$ with normal distribution $ Z\sim\mathcal{N}\left(0,\sigma^{2}\right) $, we define the random varaible $$ Y=X+Z $$

(We assume $X$,$Z$ are independent random variables)

Now, in order to estimate $ X $ by the samples $ Y $, we decide that if $ Y \geq \gamma $ then $ X=1 $, and if $ Y< \gamma $ then $ X=-1 $, where $ \gamma $ is some value.

Now, the probability of a mistake is given by:

$$ \mathbb{P}_{\text{error}}\left(\gamma\right)=\left(1-\alpha\right)-\left(1-\alpha\right)F_{Z}\left(\gamma+1\right)+\alpha F_{Z}\left(\gamma-1\right) $$

And I was supposed to find $ \gamma $ such that the error would be minimal. I did the calculations and proved that this probability accepts its minimal value for $$ \gamma=\frac{1}{2}\sigma^{2}\ln\left(\frac{1-\alpha}{\alpha}\right) $$

Now I'm asked to explain why this result makes sense. Putting the calculations aside, Im not sure why would this particular $\gamma $ lead to minimal error. I'd really appreciate an idea or explanation.

Other results I have calculated and may be important for the explanation

The density of $ Y $ : $$ f_{Y}\left(y\right)=\alpha\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{\left(y-1\right)^{2}}{2\sigma^{2}}}+\left(1-\alpha\right)\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{\left(y+1\right)^{2}}{2\sigma^{2}}} $$

Explanation for the probability of the eroor

The cases where we have determined $X$ and got a false result,So the cases are exactly where $ Y\geq\gamma\cap X=-1 $ or $ Y<\gamma\cap X=1 $.

So that

\begin{align} \mathbb{P}_{\text{error}} &=\mathbb{P}\left(Y\geq\gamma\cap X=-1\right)+\mathbb{P}\left(Y<\gamma\cap X=1\right) \\ &=\mathbb{P}\left(X+Z\geq\gamma,X=-1\right)+\mathbb{P}\left(X+Z<\gamma,X=1\right)\\ & =\mathbb{P}\left(Z\geq\gamma+1,X=-1\right)+\mathbb{P}\left(Z<\gamma-1,X=1\right) \\ &\underset{X,Z\text{ ind.}}{=}\mathbb{P}\left(X=-1\right)\mathbb{P}\left(Z\geq\gamma+1\right)+\mathbb{P}\left(X=1\right)\mathbb{P}\left(Z<\gamma-1\right) \\ &=\left(1-\alpha\right)\left(1-F_{Z}\left(\gamma+1\right)\right)+\alpha F_{Z}\left(\gamma-1\right) \\ \end{align}

Claculation of the minimum I'll differntiate with respect to $\gamma $ and then find roots:

$ \frac{d}{d\gamma}\mathbb{P}_{\text{error}}=-\left(1-\alpha\right)f_{Z}\left(\gamma+1\right)+\alpha f_{Z}\left(\gamma-1\right)=0 $

$ \alpha f_{Z}\left(\gamma-1\right)=\left(1-\alpha\right)f_{Z}\left(\gamma+1\right) $

$ \frac{\alpha}{1-\alpha}=\frac{f_{Z}\left(\gamma+1\right)}{f_{Z}\left(\gamma-1\right)}=\frac{e^{-\frac{\left(\gamma+1\right)^{2}}{2\sigma^{2}}}}{e^{-\frac{\left(\gamma-1\right)^{2}}{2\sigma^{2}}}}=e^{-\frac{\left(\gamma+1\right)^{2}}{2\sigma^{2}}+\frac{\left(\gamma-1\right)^{2}}{2\sigma^{2}}}=e^{\frac{-2\gamma-2\gamma}{2\sigma^{2}}}=e^{-\frac{2\gamma}{\sigma^{2}}} $


$ \frac{2\gamma}{\sigma^{2}}=\ln(\frac{1-\alpha}{\alpha}) $

and we have

$ \gamma=\frac{1}{2}\sigma^{2}\ln\left(\frac{1-\alpha}{\alpha}\right) $

  • 1
    $\begingroup$ can you explain your formula for the error probability? I'm a bit confused how you end up with $\gamma\pm1$ in the argument of the CDF! $\endgroup$ Dec 5 '21 at 13:49
  • $\begingroup$ @MarcusMüller Sure. $\endgroup$
    – FreeZe
    Dec 5 '21 at 14:33
  • 1
    $\begingroup$ Tried to align things a bit better, can you check whether I made a mistake when I inserted a = at the beginning of the last line? $\endgroup$ Dec 5 '21 at 15:06
  • $\begingroup$ @MarcusMüller Looks fine. $\endgroup$
    – FreeZe
    Dec 5 '21 at 15:07
  • $\begingroup$ The "why" should drop out of the actual calculation of $\gamma$, yet you do not show that calculation -- please add it in. $\endgroup$
    – TimWescott
    Dec 5 '21 at 16:20

Now I'm asked to explain why this result makes sense.

That's sort of a subjective question. One way to interpret this is to pick some cases where you already know the answer

  1. $\alpha =.5$ In this case everything is symmetric so your answer should be $\gamma = 0$. Check
  2. $\alpha = 1$. In this case your random variable is always 1 and you want $Y > \gamma$ for all values of $Y$. So the best choice for alpha is $\gamma = -\infty$ . Check.
  3. $\alpha = 0$. In this case your random variable is always 0 and you want $Y < \gamma$ for all values of $Y$. So the best choice for alpha is $\gamma = +\infty$ . Check.
  4. Low noise energy. If $\sigma$ is small, the noise has very little impact on your observation, so putting $\gamma = 0$ symmetrically between -1 and +1 makes sense. Check.
  5. It should be symmetric in $\alpha$ , i.e. $\gamma(\alpha) = -\gamma(1-\alpha)$. Check.

For anything else you can try to make up a plausibility argument or just run the numbers manually for one case.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.