# Calculating error of estimation of signal from noisy data (Explanation of result)

Given random varaible X with distribution

$$\begin{cases} \mathbb{P}\left(X=1\right)=\alpha\\ \mathbb{P}\left(X=-1\right)=1-\alpha \end{cases}$$

Where $$\alpha$$ is a given parameter($$X$$ is a binary signal), and noise $$Z$$ with normal distribution $$Z\sim\mathcal{N}\left(0,\sigma^{2}\right)$$, we define the random varaible $$Y=X+Z$$

(We assume $$X$$,$$Z$$ are independent random variables)

Now, in order to estimate $$X$$ by the samples $$Y$$, we decide that if $$Y \geq \gamma$$ then $$X=1$$, and if $$Y< \gamma$$ then $$X=-1$$, where $$\gamma$$ is some value.

Now, the probability of a mistake is given by:

$$\mathbb{P}_{\text{error}}\left(\gamma\right)=\left(1-\alpha\right)-\left(1-\alpha\right)F_{Z}\left(\gamma+1\right)+\alpha F_{Z}\left(\gamma-1\right)$$

And I was supposed to find $$\gamma$$ such that the error would be minimal. I did the calculations and proved that this probability accepts its minimal value for $$\gamma=\frac{1}{2}\sigma^{2}\ln\left(\frac{1-\alpha}{\alpha}\right)$$

Now I'm asked to explain why this result makes sense. Putting the calculations aside, Im not sure why would this particular $$\gamma$$ lead to minimal error. I'd really appreciate an idea or explanation.

## Other results I have calculated and may be important for the explanation

The density of $$Y$$ : $$f_{Y}\left(y\right)=\alpha\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{\left(y-1\right)^{2}}{2\sigma^{2}}}+\left(1-\alpha\right)\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{\left(y+1\right)^{2}}{2\sigma^{2}}}$$

Explanation for the probability of the eroor

The cases where we have determined $$X$$ and got a false result,So the cases are exactly where $$Y\geq\gamma\cap X=-1$$ or $$Y<\gamma\cap X=1$$.

So that

\begin{align} \mathbb{P}_{\text{error}} &=\mathbb{P}\left(Y\geq\gamma\cap X=-1\right)+\mathbb{P}\left(Y<\gamma\cap X=1\right) \\ &=\mathbb{P}\left(X+Z\geq\gamma,X=-1\right)+\mathbb{P}\left(X+Z<\gamma,X=1\right)\\ & =\mathbb{P}\left(Z\geq\gamma+1,X=-1\right)+\mathbb{P}\left(Z<\gamma-1,X=1\right) \\ &\underset{X,Z\text{ ind.}}{=}\mathbb{P}\left(X=-1\right)\mathbb{P}\left(Z\geq\gamma+1\right)+\mathbb{P}\left(X=1\right)\mathbb{P}\left(Z<\gamma-1\right) \\ &=\left(1-\alpha\right)\left(1-F_{Z}\left(\gamma+1\right)\right)+\alpha F_{Z}\left(\gamma-1\right) \\ \end{align}

Claculation of the minimum I'll differntiate with respect to $$\gamma$$ and then find roots:

$$\frac{d}{d\gamma}\mathbb{P}_{\text{error}}=-\left(1-\alpha\right)f_{Z}\left(\gamma+1\right)+\alpha f_{Z}\left(\gamma-1\right)=0$$

$$\alpha f_{Z}\left(\gamma-1\right)=\left(1-\alpha\right)f_{Z}\left(\gamma+1\right)$$

$$\frac{\alpha}{1-\alpha}=\frac{f_{Z}\left(\gamma+1\right)}{f_{Z}\left(\gamma-1\right)}=\frac{e^{-\frac{\left(\gamma+1\right)^{2}}{2\sigma^{2}}}}{e^{-\frac{\left(\gamma-1\right)^{2}}{2\sigma^{2}}}}=e^{-\frac{\left(\gamma+1\right)^{2}}{2\sigma^{2}}+\frac{\left(\gamma-1\right)^{2}}{2\sigma^{2}}}=e^{\frac{-2\gamma-2\gamma}{2\sigma^{2}}}=e^{-\frac{2\gamma}{\sigma^{2}}}$$

So

$$\frac{2\gamma}{\sigma^{2}}=\ln(\frac{1-\alpha}{\alpha})$$

and we have

$$\gamma=\frac{1}{2}\sigma^{2}\ln\left(\frac{1-\alpha}{\alpha}\right)$$

• can you explain your formula for the error probability? I'm a bit confused how you end up with $\gamma\pm1$ in the argument of the CDF! Dec 5 '21 at 13:49
• @MarcusMüller Sure. Dec 5 '21 at 14:33
• Tried to align things a bit better, can you check whether I made a mistake when I inserted a = at the beginning of the last line? Dec 5 '21 at 15:06
• @MarcusMüller Looks fine. Dec 5 '21 at 15:07
• The "why" should drop out of the actual calculation of $\gamma$, yet you do not show that calculation -- please add it in. Dec 5 '21 at 16:20

1. $$\alpha =.5$$ In this case everything is symmetric so your answer should be $$\gamma = 0$$. Check
2. $$\alpha = 1$$. In this case your random variable is always 1 and you want $$Y > \gamma$$ for all values of $$Y$$. So the best choice for alpha is $$\gamma = -\infty$$ . Check.
3. $$\alpha = 0$$. In this case your random variable is always 0 and you want $$Y < \gamma$$ for all values of $$Y$$. So the best choice for alpha is $$\gamma = +\infty$$ . Check.
4. Low noise energy. If $$\sigma$$ is small, the noise has very little impact on your observation, so putting $$\gamma = 0$$ symmetrically between -1 and +1 makes sense. Check.
5. It should be symmetric in $$\alpha$$ , i.e. $$\gamma(\alpha) = -\gamma(1-\alpha)$$. Check.