1

Assuming that $f_{X|D}(x|d)=e^{\alpha d x - \psi(d)}f_0(x)$ for some constant $\alpha$ such that it is a valid pdf for every value of $d$, i want to establish that: $$\mathbb{E}[D|X=x]=\frac{1}{\alpha}\frac{\partial}{\partial x}\lambda(x)\quad \text{, where}\quad \lambda(x)=\log\frac{f_X(x)}{f_0(x)}$$

*There's no missing information. It's from a collection of Information Theory problems.

Oriol B
  • 198
  • 1
  • 7

1 Answers1

1

We can compute the marginal PDF of $X$ as follows: \begin{align} f_X(x)=\int_\delta f_{X|D}(x|\delta)f_D(\delta)d\delta = f_0(x) \int_\delta e^{\alpha \delta x - \psi(\delta)} f_D(\delta)d\delta \end{align}

We can express the previous equation in terms of $\lambda(x)$ by dividing both sides by $f_0(x)$ and applying the natural logarithm.

\begin{align} \lambda(x) := \log \frac{f_X(x)}{f_0(x)} = \log \left[ \int_\delta e^{\alpha \delta x - \psi(\delta)} f_D(\delta)d\delta\right] \end{align}

Now, if we derive w.r.t the parameter $x$, we obtain the following equality:

\begin{align} \frac{\partial}{\partial x} \lambda (x) = \frac{\int_\delta \alpha \delta e^{\alpha \delta x - \psi(\delta)} f_D(\delta)d\delta}{\int_\delta e^{\alpha \delta x - \psi(\delta)} f_D(\delta)d\delta}= \alpha \frac{\int_\delta \delta e^{\alpha \delta x - \psi(\delta)} f_D(\delta)d\delta}{\int_\delta e^{\alpha \delta x - \psi(\delta)} f_D(\delta)d\delta} \end{align}

With this equality, we are now able to express the MMSE estimator of $D$ in terms of $\lambda(x)$:

\begin{align} \mathbb{E}[D|X=x]&=\int_{\delta}\delta f_{D|X}(\delta|x)d\delta=\int_{\delta}\delta\frac{f_{X|D}(x|\delta)f_D (\delta)}{f_X(x)}d\delta=\frac{\int_\delta \delta f_0(x)e^{\alpha \delta x - \psi(\delta)}f_D(\delta) d\delta}{f_X(x)} = \\ &= \frac{\int_\delta \delta f_0(x)e^{\alpha \delta x - \psi(\delta)}f_D(\delta) d\delta}{\int_\delta f_0(x) e^{\alpha \delta x - \psi(\delta)} f_D(\delta)d\delta}=\frac{1}{\alpha}\frac{\partial}{\partial x} \lambda (x) \end{align}

Oriol B
  • 198
  • 1
  • 7