While reading this paper https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5075875, I found that the the symmetric capacity of BDMC $W$ is given by

$$I(W) = \sum_{y\in Y}\sum_{x\in X}\frac{1}{2}W(y|x)\log_{2}\Bigg(\frac{W(y|x)}{0.5W(y|0)+0.5W(y|1)}\Bigg)$$ I am unable to understand how this equation is derived. Usually, the information in a channel is given by $$I(X=x_{i}) = \log_{2}(\frac{1}{P(x_{i})})$$ where $P$ is the probability operator. However, what I see is that the first equation is that of entropy because it is averaging over the space of $X$ and $Y$. How is this expression equivalent to $I$?

Note that $W(y|x)$ is the transition probability, that is, when $X$ transmit a bit, and $Y$ receives the flipped bit, then the probability of this phenomena happening is $W(y|x)$.

  • $\begingroup$ Your second equation is the information in $x_i$, independent of any channel. I suggest you to start by reading the first few chapters of an information theory book, such as Cover or MacKay. $\endgroup$ – MBaz Apr 19 '19 at 14:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.