2
$\begingroup$

I am designing a function $A(x)$ that map $x\to \{0,1\}$. The function $A$ can be expressed as $$A(x)=\begin{cases} 1 \text{ if $p_1(x) \ge p_2(x)$}\\ 0 \text{ otherwises} \end{cases} $$ where $p_1$ and $p_2$ are Gaussian distribution:

$p_1(x)=\frac{1}{\sqrt{2\pi}\sigma_1}\exp\left( {-\frac{(x-\mu_1)^2}{2\sigma_1^2}} \right)$;

$p_2(x)=\frac{1}{\sqrt{2\pi}\sigma_2}\exp\left( {-\frac{(x-\mu_2)^2}{2\sigma_2^2}} \right)$.

Could you help me to derive a short form of $A(x)$ ?

This is my solution. However, it maybe wrong

I derived the bellow eq. $$A(x)=\begin{cases} 1 \text{ if $\log \frac{p_1(x)}{p_2(x)}>0$}\\ 0 \text{ otherwises} \end{cases} $$ where $\log(e)=1$

We have $$\log \frac{p_1(x)}{p_2(x)}=\log \frac{\sigma_2}{\sigma_1}+\left( {\frac{x-\mu_2}{\sqrt{2}\sigma_2}} \right)^2-\left( {\frac{x-\mu_1}{\sqrt{2}\sigma_1}} \right)^2$$

Hence, my final solution is $$A(x)=\begin{cases} 1 \text{ if $\left (\log \frac{\sigma_2}{\sigma_1}+\left( {\frac{x-\mu_2}{\sqrt{2}\sigma_2}} \right)^2-\left( {\frac{x-\mu_1}{\sqrt{2}\sigma_1}} \right)^2 \right) \ge 0$}\\ 0 \text{ otherwises} \end{cases} $$

Thanks in advance

$\endgroup$
7
  • $\begingroup$ You somehow lost the square in your "final solution". $\endgroup$
    – Matt L.
    Nov 24, 2015 at 14:28
  • $\begingroup$ because $\log(\exp(-x^2))=-2x$,right? $\endgroup$
    – Jame
    Nov 24, 2015 at 14:36
  • $\begingroup$ No, $\log(\exp(-x^2))=-x^2$. $\endgroup$
    – Matt L.
    Nov 24, 2015 at 15:01
  • $\begingroup$ The $\sqrt{2}$ in the denominators of your final formula should be a $2$. Then it's correct, but you should work it out to become a condition on $x$. As it stands, it can't really be evaluated that easily. $\endgroup$
    – Matt L.
    Nov 24, 2015 at 16:25
  • $\begingroup$ Because I put square out side, thus, inside the denominators only remains $\sqrt{2}$, is it correct? What does it means of "out to become a condition on $x$", In my case, $x,\sigma_1,\sigma_2,\mu_1,\mu_2$ are given $\endgroup$
    – Jame
    Nov 24, 2015 at 16:31

1 Answer 1

3
$\begingroup$

Your final condition is correct, but it doesn't show explicitly for which values of $x$ it is satisfied. This can be worked out as follows. From what you have it follows that $p_1(x)\ge p_2(x)$ is equivalent to

$$0\ge x^2(\sigma_2^2-\sigma_1^2)-2x(\mu_1\sigma_2^2-\mu_2\sigma_1^2)+\mu_1^2\sigma_2^2-\mu_2^2\sigma_1^2-2\sigma_1^2\sigma_2^2\log\left(\frac{\sigma_2}{\sigma_1}\right)\tag{1}$$

Let's first consider the case $\sigma_1=\sigma_2$. From $(1)$ it follows that in this case the condition on $x$ becomes

$$x\ge \frac{\mu_1^2-\mu_2^2}{2(\mu_1-\mu_2)} = \frac{\mu_1+\mu_2}{2}\tag{2}$$

where we assume $\mu_1\neq\mu_2$ because otherwise $p_1(x)=p_2(x)$, which makes no sense.

In the case $\sigma_1\neq\sigma_2$, we have to consider $(1)$ as a quadratic inequality in $x$, which is satisfied with equality for the following values of $x$:

$$\begin{align}x_1&=\frac{a}{2}-\sqrt{\frac{a^2}{4}-b}\\ x_2&=\frac{a}{2}+\sqrt{\frac{a^2}{4}-b}\end{align}\tag{3}$$

with

$$\begin{align}a&=\frac{2(\mu_1\sigma_2^2-\mu_2\sigma_1^2)}{\sigma_2^2-\sigma_1^2}\\ b&=\frac{\mu_1^2\sigma_2^2-\mu_2^2\sigma_1^2-2\sigma_1^2\sigma_2^2\log\left(\frac{\sigma_2}{\sigma_1}\right)}{\sigma_2^2-\sigma_1^2}\end{align}\tag{4}$$

Depending on the sign of $\sigma_2-\sigma_1$ we have two conditions equivalent to $p_1(x)\ge p_2(x)$:

$$\begin{align}\sigma_2>\sigma_1:\quad & x_1\le x\le x_2\\ \sigma_2<\sigma_1:\quad & x\le x_1\text{ or }x\ge x_2\end{align}\tag{5}$$

The inequalities in $(5)$ state explicit conditions on the variable $x$ that can be easily evaluated.

EDIT: As suggested by Dilip Sarwate in a comment it can be instructive to distinguish another special case: $\sigma_1\neq\sigma_2$ and $\mu_1=\mu_2=\mu$. In that case the limits $x_1$ and $x_2$ of the decision regions given by $(3)$ lie symmetrical to the common mean $\mu$, and $(3)$ simplifies to

$$x_{1,2}=\mu\pm\sqrt{\frac{2\sigma_1^2\sigma_2^2\log\left(\frac{\sigma_2}{\sigma_1}\right)}{\sigma_2^2-\sigma_1^2}}$$

Of course, the conditions $(5)$ remain valid.

$\endgroup$
5
  • $\begingroup$ +1 I improved (2) a little; I hope you approve. $\endgroup$ Nov 24, 2015 at 17:12
  • $\begingroup$ @DilipSarwate: Very good, silly me ... $\endgroup$
    – Matt L.
    Nov 24, 2015 at 17:13
  • 1
    $\begingroup$ Nice, guys!!!!! $\endgroup$
    – Peter K.
    Nov 24, 2015 at 17:16
  • 1
    $\begingroup$ Actually, if I may suggest: there are two special cases where the answers are simpler, and perhaps both should be considered separately before looking at the more general answer. (1) $\sigma_1=\sigma_2, \mu_1 \neq \mu_2$ where the threshold is $\frac{\mu_1+\mu_2}{2}$; drawing a sketch of the two pdfs reveals, without any algebra why the threshold must lie halfway between the means, and (2) $\sigma_1 \neq \sigma_2, \mu_1 = \mu_2$ in which case the decision is to choose one hypothesis if $x$ is far away from the common mean, and the other hypothesis if $x$ is near the common mean; again sketch! $\endgroup$ Nov 24, 2015 at 17:20
  • $\begingroup$ @DilipSarwate: I've added that second special case to my answer. $\endgroup$
    – Matt L.
    Nov 24, 2015 at 20:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.