When channels are in a cascade and are independent is the capacity the sum?

Just wondering if two channels are independent then is the overall capacity, the sum of the individual capacities?

For example suppose I have two BEC where the probability of erasure is $$\varepsilon$$ and $$\varphi$$ respectively.

The individual capacities are then $$1- \varepsilon$$ and $$1-\varphi$$ respectively. Now if these channels were independent and in a cascade would the overall capacity be $$1- \varepsilon$$ + $$1 - \varphi$$?

Best regards and thanks for indulging me.

Just wondering if two channels are independent then is the overall capacity, the sum of the individual capacities?

Yes. That directly results from the very definition of independence and of capacity!

Now if these channels were independent and in a cascade would the overall capacity be 1−ε + 1−φ?

Oh! That's something different! You don't have two independent channels that you use, you have a single channel that is composed of a series of these two.

The first channel's mutual information is at most the original source's entropy (in fact, it's lower, as you notice, $$1-\epsilon$$). The second channel again has a mutual information that's at most as high as its input entropy, and that is already limited by the mutual information of the first channel.

Hint: try to make a probabilities "table" for outputs given inputs, and try to model this new combined channel to find out the true capacity. It's easy!

• Firstly thank you for your answer, I understand the reasoning you've given behind the limiting of mutual information!! However, I'm not sure on the probabilities, I think I have P(X|Z) not sure on P(X,Z) at all struggling to be honest. Dec 1 '19 at 17:03
• as said, write it down as a table. Dec 1 '19 at 19:00
• I made the table but it differs from the minimum of each individual channel. Dec 4 '19 at 10:39
• The minimum is just a lower bound, so that's not surprising at all. Dec 4 '19 at 10:40
• Ah, thank you for that ! Dec 4 '19 at 10:42

Let the first channel, channel A, have capacity $$C_A=1-\varepsilon$$ and the second channel, channel B, have capacity $$C_B=1-\varphi$$. Now put them in cascade, that is, first channel A then channel B. The resulting capacity of the cascaded channel will be $$\text{min}(C_A, C_B)$$. You can figure this out by thinking about the information flow like water through pipes. For example, if you're able to push $$X$$ bits through channel A but only $$X-10$$ bits through channel B, well then the total capacity will be limited by the smallest capacity channel in the cascaded system. Hope this helps!

• Hi there I've made a table like Marcus suggested and got a value of 0.80762 for the mutual information. When considering the min of both channels I get a value of 0.85856. I can't understand where I have gone wrong :( Dec 4 '19 at 10:38