I'm trying to derive the equation of error probability in orthogonal signaling. Here what I've tried:
Let $\vec{s_1} = (\sqrt{E_b} , 0)$ and $\vec{s_2} = (0, \sqrt{E_b} )$ be vector representation of $s_1$ and $s_2$. We know that $$f(\vec{y}|\vec{s_m}) = (\frac{1}{\sqrt{\pi N_0}})^2e^{-\frac{|\vec{y} - \vec{s_m}|^2}{N_0}} , \ m = 1,2$$ According to the Bayes' law, we can write $$P(\vec{s_m}|\vec{y}) = \frac{f(\vec{y}|\vec{s_m})P(\vec{s_m})}{f(\vec{y})}, m = 1,2$$Let $P(\vec{s_1}) = p_1$ and $P(\vec{s_1}) = p_2$. Using MAP rule, we want to find $m$ such that $P(\vec{s_m}|\vec{y})$ be maximized. So $\vec{s_1}$ is chosen if $$P(\vec{s_1}|\vec{y}) \gt P(\vec{s_2}|\vec{y})$$ and otherwise $\vec{s_2}$. After simplification we get $$\frac{N_0}{\sqrt{E_b}} \ln(\frac{p_1}{p_2}) \gt -2(y_1 - y_2)$$ Let $a = \frac{-N_0 \ln(\frac{p_1}{p_2})}{2\sqrt{E_b}}$ and note that $y_1 - y_2$ is a Gaussian random variable with mean $\sqrt{E_b}$ and variance $\sigma^2 = N_0$. So the probability of error is $$P_e = p_1\int_{-\infty}^a \frac{1}{\sqrt{2\pi N_0}}e^{-\frac{(z-\sqrt{E_b})^2}{2N_0}}dz + p_2\int_{a}^{+\infty} \frac{1}{\sqrt{2\pi N_0}}e^{-\frac{(z-\sqrt{E_b})^2}{2N_0}}dz$$ which simplifies to $$P_e = p_1\int_{-\infty}^{\frac{a - \sqrt{E_b}}{\sqrt{N_0}}} \frac{1}{\sqrt{2\pi}}e^{-\frac{u^2}{2}}du + p_2\int_{\frac{a - \sqrt{E_b}}{\sqrt{N_0}}}^{+\infty} \frac{1}{\sqrt{2\pi}}e^{-\frac{u^2}{2}}du = p_1Q(\frac{\sqrt{E_b} - a}{\sqrt{N_0}}) + p_2Q(\frac{a - \sqrt{E_b} }{\sqrt{N_0}})$$ Is this result correct? I couldn't find a source which includes error probability for the orthogonal signaling when prior probabilities are unequal.
Edit: Thanks to Dilip Sarwate's comment, the second exponent should be $$-\frac{(z+\sqrt{E_b})^2}{2N_0}$$ So the final answer is $$P_e = p_1Q(\frac{\sqrt{E_b} - a}{\sqrt{N_0}}) + p_2Q(\frac{\sqrt{E_b} + a }{\sqrt{N_0}})$$ Where $a = \frac{-N_0 \ln(\frac{p_1}{p_2})}{2\sqrt{E_b}}$. We can simplify this formula and get $$P_e = p_1Q(\sqrt{\frac{E_b}{N_0}} - \frac{\sqrt{N_0}}{2\sqrt{E_b}}\ln(\frac{p_2}{p_1})) + p_2Q(\sqrt{\frac{E_b}{N_0}} +\frac{\sqrt{N_0}}{2\sqrt{E_b}}\ln(\frac{p_2}{p_1}))$$ which agrees with Dilip Sarwate's answer.