1
$\begingroup$

I am designing a function $A(x)$ that map $x\to \{0,1\}$. The function $A$ can be expressed as $$A(x)=\begin{cases} 1 \text{ if $p_1(x) \ge p_2(x)$}\\ 0 \text{ otherwises} \end{cases} $$ where $p_1$ and $p_2$ are Gaussian distribution:

$p_1(x)=\frac{1}{\sqrt{2\pi}\sigma_1}\exp\left( {-\frac{(x-\mu_1)^2}{2\sigma_1^2}} \right)$;

$p_2(x)=\frac{1}{\sqrt{2\pi}\sigma_2}\exp\left( {-\frac{(x-\mu_2)^2}{2\sigma_2^2}} \right)$.

Could you help me to derive a short form of $A(x)$ ?

This is my solution. However, it maybe wrong

I derived the bellow eq. $$A(x)=\begin{cases} 1 \text{ if $\log \frac{p_1(x)}{p_2(x)}>0$}\\ 0 \text{ otherwises} \end{cases} $$ where $\log(e)=1$

We have $$\log \frac{p_1(x)}{p_2(x)}=\log \frac{\sigma_2}{\sigma_1}+\left( {\frac{x-\mu_2}{\sqrt{2}\sigma_2}} \right)^2-\left( {\frac{x-\mu_1}{\sqrt{2}\sigma_1}} \right)^2$$

Hence, my final solution is $$A(x)=\begin{cases} 1 \text{ if $\left (\log \frac{\sigma_2}{\sigma_1}+\left( {\frac{x-\mu_2}{\sqrt{2}\sigma_2}} \right)^2-\left( {\frac{x-\mu_1}{\sqrt{2}\sigma_1}} \right)^2 \right) \ge 0$}\\ 0 \text{ otherwises} \end{cases} $$

Thanks in advance

$\endgroup$
  • $\begingroup$ You somehow lost the square in your "final solution". $\endgroup$ – Matt L. Nov 24 '15 at 14:28
  • $\begingroup$ because $\log(\exp(-x^2))=-2x$,right? $\endgroup$ – Jame Nov 24 '15 at 14:36
  • $\begingroup$ No, $\log(\exp(-x^2))=-x^2$. $\endgroup$ – Matt L. Nov 24 '15 at 15:01
  • $\begingroup$ The $\sqrt{2}$ in the denominators of your final formula should be a $2$. Then it's correct, but you should work it out to become a condition on $x$. As it stands, it can't really be evaluated that easily. $\endgroup$ – Matt L. Nov 24 '15 at 16:25
  • $\begingroup$ Because I put square out side, thus, inside the denominators only remains $\sqrt{2}$, is it correct? What does it means of "out to become a condition on $x$", In my case, $x,\sigma_1,\sigma_2,\mu_1,\mu_2$ are given $\endgroup$ – Jame Nov 24 '15 at 16:31
3
$\begingroup$

Your final condition is correct, but it doesn't show explicitly for which values of $x$ it is satisfied. This can be worked out as follows. From what you have it follows that $p_1(x)\ge p_2(x)$ is equivalent to

$$0\ge x^2(\sigma_2^2-\sigma_1^2)-2x(\mu_1\sigma_2^2-\mu_2\sigma_1^2)+\mu_1^2\sigma_2^2-\mu_2^2\sigma_1^2-2\sigma_1^2\sigma_2^2\log\left(\frac{\sigma_2}{\sigma_1}\right)\tag{1}$$

Let's first consider the case $\sigma_1=\sigma_2$. From $(1)$ it follows that in this case the condition on $x$ becomes

$$x\ge \frac{\mu_1^2-\mu_2^2}{2(\mu_1-\mu_2)} = \frac{\mu_1+\mu_2}{2}\tag{2}$$

where we assume $\mu_1\neq\mu_2$ because otherwise $p_1(x)=p_2(x)$, which makes no sense.

In the case $\sigma_1\neq\sigma_2$, we have to consider $(1)$ as a quadratic inequality in $x$, which is satisfied with equality for the following values of $x$:

$$\begin{align}x_1&=\frac{a}{2}-\sqrt{\frac{a^2}{4}-b}\\ x_2&=\frac{a}{2}+\sqrt{\frac{a^2}{4}-b}\end{align}\tag{3}$$

with

$$\begin{align}a&=\frac{2(\mu_1\sigma_2^2-\mu_2\sigma_1^2)}{\sigma_2^2-\sigma_1^2}\\ b&=\frac{\mu_1^2\sigma_2^2-\mu_2^2\sigma_1^2-2\sigma_1^2\sigma_2^2\log\left(\frac{\sigma_2}{\sigma_1}\right)}{\sigma_2^2-\sigma_1^2}\end{align}\tag{4}$$

Depending on the sign of $\sigma_2-\sigma_1$ we have two conditions equivalent to $p_1(x)\ge p_2(x)$:

$$\begin{align}\sigma_2>\sigma_1:\quad & x_1\le x\le x_2\\ \sigma_2<\sigma_1:\quad & x\le x_1\text{ or }x\ge x_2\end{align}\tag{5}$$

The inequalities in $(5)$ state explicit conditions on the variable $x$ that can be easily evaluated.

EDIT: As suggested by Dilip Sarwate in a comment it can be instructive to distinguish another special case: $\sigma_1\neq\sigma_2$ and $\mu_1=\mu_2=\mu$. In that case the limits $x_1$ and $x_2$ of the decision regions given by $(3)$ lie symmetrical to the common mean $\mu$, and $(3)$ simplifies to

$$x_{1,2}=\mu\pm\sqrt{\frac{2\sigma_1^2\sigma_2^2\log\left(\frac{\sigma_2}{\sigma_1}\right)}{\sigma_2^2-\sigma_1^2}}$$

Of course, the conditions $(5)$ remain valid.

$\endgroup$
  • $\begingroup$ +1 I improved (2) a little; I hope you approve. $\endgroup$ – Dilip Sarwate Nov 24 '15 at 17:12
  • $\begingroup$ @DilipSarwate: Very good, silly me ... $\endgroup$ – Matt L. Nov 24 '15 at 17:13
  • 1
    $\begingroup$ Nice, guys!!!!! $\endgroup$ – Peter K. Nov 24 '15 at 17:16
  • 1
    $\begingroup$ Actually, if I may suggest: there are two special cases where the answers are simpler, and perhaps both should be considered separately before looking at the more general answer. (1) $\sigma_1=\sigma_2, \mu_1 \neq \mu_2$ where the threshold is $\frac{\mu_1+\mu_2}{2}$; drawing a sketch of the two pdfs reveals, without any algebra why the threshold must lie halfway between the means, and (2) $\sigma_1 \neq \sigma_2, \mu_1 = \mu_2$ in which case the decision is to choose one hypothesis if $x$ is far away from the common mean, and the other hypothesis if $x$ is near the common mean; again sketch! $\endgroup$ – Dilip Sarwate Nov 24 '15 at 17:20
  • $\begingroup$ @DilipSarwate: I've added that second special case to my answer. $\endgroup$ – Matt L. Nov 24 '15 at 20:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.