Just wondering if two channels are independent then is the overall capacity, the sum of the individual capacities?
For example suppose I have two BEC where the probability of erasure is $\varepsilon$ and $\varphi$ respectively.
The individual capacities are then $1- \varepsilon$ and $1-\varphi$ respectively. Now if these channels were independent and in a cascade would the overall capacity be $1- \varepsilon$ + $1 - \varphi$?
Best regards and thanks for indulging me.
Just wondering if two channels are independent then is the overall capacity, the sum of the individual capacities?
Yes. That directly results from the very definition of independence and of capacity!
Now if these channels were independent and in a cascade would the overall capacity be 1−ε + 1−φ?
Oh! That's something different! You don't have two independent channels that you use, you have a single channel that is composed of a series of these two.
So, no, the sum can't be right. Think about this:
The first channel's mutual information is at most the original source's entropy (in fact, it's lower, as you notice, $1-\epsilon$). The second channel again has a mutual information that's at most as high as its input entropy, and that is already limited by the mutual information of the first channel.
Hint: try to make a probabilities "table" for outputs given inputs, and try to model this new combined channel to find out the true capacity. It's easy!
Let the first channel, channel A, have capacity $C_A=1-\varepsilon$ and the second channel, channel B, have capacity $C_B=1-\varphi$. Now put them in cascade, that is, first channel A then channel B. The resulting capacity of the cascaded channel will be $\text{min}(C_A, C_B)$. You can figure this out by thinking about the information flow like water through pipes. For example, if you're able to push $X$ bits through channel A but only $X-10$ bits through channel B, well then the total capacity will be limited by the smallest capacity channel in the cascaded system. Hope this helps!
