1
$\begingroup$

Assuming that $f_{X|D}(x|d)=e^{\alpha d x - \psi(d)}f_0(x)$ for some constant $\alpha$ such that it is a valid pdf for every value of $d$, i want to establish that: $$\mathbb{E}[D|X=x]=\frac{1}{\alpha}\frac{\partial}{\partial x}\lambda(x)\quad \text{, where}\quad \lambda(x)=\log\frac{f_X(x)}{f_0(x)}$$

*There's no missing information. It's from a collection of Information Theory problems.

$\endgroup$
1
$\begingroup$

We can compute the marginal PDF of $X$ as follows: \begin{align} f_X(x)=\int_\delta f_{X|D}(x|\delta)f_D(\delta)d\delta = f_0(x) \int_\delta e^{\alpha \delta x - \psi(\delta)} f_D(\delta)d\delta \end{align}

We can express the previous equation in terms of $\lambda(x)$ by dividing both sides by $f_0(x)$ and applying the natural logarithm.

\begin{align} \lambda(x) := \log \frac{f_X(x)}{f_0(x)} = \log \left[ \int_\delta e^{\alpha \delta x - \psi(\delta)} f_D(\delta)d\delta\right] \end{align}

Now, if we derive w.r.t the parameter $x$, we obtain the following equality:

\begin{align} \frac{\partial}{\partial x} \lambda (x) = \frac{\int_\delta \alpha \delta e^{\alpha \delta x - \psi(\delta)} f_D(\delta)d\delta}{\int_\delta e^{\alpha \delta x - \psi(\delta)} f_D(\delta)d\delta}= \alpha \frac{\int_\delta \delta e^{\alpha \delta x - \psi(\delta)} f_D(\delta)d\delta}{\int_\delta e^{\alpha \delta x - \psi(\delta)} f_D(\delta)d\delta} \end{align}

With this equality, we are now able to express the MMSE estimator of $D$ in terms of $\lambda(x)$:

\begin{align} \mathbb{E}[D|X=x]&=\int_{\delta}\delta f_{D|X}(\delta|x)d\delta=\int_{\delta}\delta\frac{f_{X|D}(x|\delta)f_D (\delta)}{f_X(x)}d\delta=\frac{\int_\delta \delta f_0(x)e^{\alpha \delta x - \psi(\delta)}f_D(\delta) d\delta}{f_X(x)} = \\ &= \frac{\int_\delta \delta f_0(x)e^{\alpha \delta x - \psi(\delta)}f_D(\delta) d\delta}{\int_\delta f_0(x) e^{\alpha \delta x - \psi(\delta)} f_D(\delta)d\delta}=\frac{1}{\alpha}\frac{\partial}{\partial x} \lambda (x) \end{align}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.