0
$\begingroup$

Just wondering if two channels are independent then is the overall capacity, the sum of the individual capacities?

For example suppose I have two BEC where the probability of erasure is $\varepsilon$ and $\varphi$ respectively.

The individual capacities are then $1- \varepsilon$ and $1-\varphi$ respectively. Now if these channels were independent and in a cascade would the overall capacity be $1- \varepsilon$ + $1 - \varphi$?

Best regards and thanks for indulging me.

Improve this question
$\endgroup$
0
$\begingroup$

Just wondering if two channels are independent then is the overall capacity, the sum of the individual capacities?

Yes. That directly results from the very definition of independence and of capacity!

Now if these channels were independent and in a cascade would the overall capacity be 1−ε + 1−φ?

Oh! That's something different! You don't have two independent channels that you use, you have a single channel that is composed of a series of these two.

So, no, the sum can't be right. Think about this:

The first channel's mutual information is at most the original source's entropy (in fact, it's lower, as you notice, $1-\epsilon$). The second channel again has a mutual information that's at most as high as its input entropy, and that is already limited by the mutual information of the first channel.

Hint: try to make a probabilities "table" for outputs given inputs, and try to model this new combined channel to find out the true capacity. It's easy!

Improve this answer
$\endgroup$
5
  • $\begingroup$ Firstly thank you for your answer, I understand the reasoning you've given behind the limiting of mutual information!! However, I'm not sure on the probabilities, I think I have P(X|Z) not sure on P(X,Z) at all struggling to be honest. $\endgroup$
    – Dead_Ling0
    Dec 1 '19 at 17:03
  • $\begingroup$ as said, write it down as a table. $\endgroup$
    – Marcus Müller
    Dec 1 '19 at 19:00
  • $\begingroup$ I made the table but it differs from the minimum of each individual channel. $\endgroup$
    – Dead_Ling0
    Dec 4 '19 at 10:39
  • $\begingroup$ The minimum is just a lower bound, so that's not surprising at all. $\endgroup$
    – Marcus Müller
    Dec 4 '19 at 10:40
  • $\begingroup$ Ah, thank you for that ! $\endgroup$
    – Dead_Ling0
    Dec 4 '19 at 10:42
0
$\begingroup$

Let the first channel, channel A, have capacity $C_A=1-\varepsilon$ and the second channel, channel B, have capacity $C_B=1-\varphi$. Now put them in cascade, that is, first channel A then channel B. The resulting capacity of the cascaded channel will be $\text{min}(C_A, C_B)$. You can figure this out by thinking about the information flow like water through pipes. For example, if you're able to push $X$ bits through channel A but only $X-10$ bits through channel B, well then the total capacity will be limited by the smallest capacity channel in the cascaded system. Hope this helps!

Improve this answer
$\endgroup$
1
  • $\begingroup$ Hi there I've made a table like Marcus suggested and got a value of 0.80762 for the mutual information. When considering the min of both channels I get a value of 0.85856. I can't understand where I have gone wrong :( $\endgroup$
    – Dead_Ling0
    Dec 4 '19 at 10:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.