4
$\begingroup$

If I understand correctly, in AWGN channel, for a given SNR, there is a code for $M$ input codeword $\mathbf{X}_{i, 1 \leq i \leq M}$ that $$\lim_{n \to \infty} R=\lim_{n \to \infty}\frac{\log(M)}{n} =\log(1+SNR)$$

(where $n$ is the length of $\mathbf{X}_i$) if $\mathbf{X}_i$ is drawn from multivariate Normal distribution.

enter image description here

However, I don't know if $\mathbf{X}_i$ is not Normal, does the Shannon channel coding theorem hold ?

Improve this question
$\endgroup$
6
  • $\begingroup$ AFAIK, the derivation of the formula $C=\log_2(1+\mathrm{SNR})$ uses the fact that the input is normally distributed. I don't know if there is a more generic proof, though. Is that what you are asking for? $\endgroup$ – Tendero Jan 23 '18 at 14:16
  • $\begingroup$ You could think about it this way: every channel+encoder+decoder system has a capacity. When the noise is Gaussian and the input distribution is also Gaussian, then the capacity is the formula you provided. When these things change, the formula for the capacity will also change. $\endgroup$ – MBaz Jan 23 '18 at 18:58
  • $\begingroup$ @Tendero yes the formula is given by taking Normal inputs. I want to ask the inverse that if the input is not Normal, can we prove that the rate is always lower the $C$ ? Because $C$ is taken by maximizing over all input distribution. $\endgroup$ – Sabrina Cantu Jan 27 '18 at 11:49
  • $\begingroup$ @MBaz thanks, I wanted to talk about the case that the rate is maximized over all possible input distributions and all possible encoder-decoder pairs. $\endgroup$ – Sabrina Cantu Jan 27 '18 at 11:50
  • $\begingroup$ @SabrinaCantu If the distribution is not normal, then the maximum rate is less than $C$. $\endgroup$ – MBaz Jan 27 '18 at 17:13
1
$\begingroup$

Wikipedia gives the more general definition. Channel capacity is defined using Information Theory. If you are unfamiliar with that, it will be hard to explain. The channel capacity is always the same for a fixed communications channel (meaning the statistics are fixed).

The more general definition of the capacity $C$ is $$ C = \sup_{p_X(x)} \mathbf{I}(\mathbf{X};\mathbf{Y}) $$ where $\mathbf{I}$ denotes the Mutual Information, $\mathbf{X}$ is the input to the channel and $\mathbf{Y}$ is the output of the channel, and the supremum is taken over all possible input distributions $p_X(x)$.

The equation that you have provided is specific to the AWGN channel. The plots are demonstrating the performance ceiling that exists because you have chosen not to use a Gaussian input sequence (for convenience usually). Notice that it follows the capacity curve to a certain limit but is then limited by the constellation size.

Codewords are relevant when trying to reach capacity, but not generally applicable in the statement of what capacity is. In other words, the channel capacity is independent of the codewords chosen.

Improve this answer
$\endgroup$
2
  • $\begingroup$ thanks. As your equation has stated, the capacity is taken by maximization over all possible input distribution. Specifically, in AWGN channel, the capacity is given by the equation is my question by using Gaussian inputs. My question is in the another senses, if I do not take Gaussian inputs, can I be sure that whatever I do, I never reach the channel capacity ? The proof in the original paper of Shannon does not follow this direction (if I understand it correctly). $\endgroup$ – Sabrina Cantu Jan 27 '18 at 11:43
  • $\begingroup$ Maybe I'm not understanding the question exactly then. Perhaps you could expand your text and try to explain better. As it is currently stated, it seems that you are asking if the input distribution chosen changes the capacity. It clearly does not affect the capacity because of the definition of capacity. Are you asking for a quantification of how much "loss" in capacity you get by altering the input distribution? $\endgroup$ – hops Jan 27 '18 at 21:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.