4
$\begingroup$

If I understand correctly, in AWGN channel, for a given SNR, there is a code for $M$ input codeword $\mathbf{X}_{i, 1 \leq i \leq M}$ that $$\lim_{n \to \infty} R=\lim_{n \to \infty}\frac{\log(M)}{n} =\log(1+SNR)$$

(where $n$ is the length of $\mathbf{X}_i$) if $\mathbf{X}_i$ is drawn from multivariate Normal distribution.

enter image description here

However, I don't know if $\mathbf{X}_i$ is not Normal, does the Shannon channel coding theorem hold ?

$\endgroup$
6
  • $\begingroup$ AFAIK, the derivation of the formula $C=\log_2(1+\mathrm{SNR})$ uses the fact that the input is normally distributed. I don't know if there is a more generic proof, though. Is that what you are asking for? $\endgroup$
    – Tendero
    Commented Jan 23, 2018 at 14:16
  • $\begingroup$ You could think about it this way: every channel+encoder+decoder system has a capacity. When the noise is Gaussian and the input distribution is also Gaussian, then the capacity is the formula you provided. When these things change, the formula for the capacity will also change. $\endgroup$
    – MBaz
    Commented Jan 23, 2018 at 18:58
  • $\begingroup$ @Tendero yes the formula is given by taking Normal inputs. I want to ask the inverse that if the input is not Normal, can we prove that the rate is always lower the $C$ ? Because $C$ is taken by maximizing over all input distribution. $\endgroup$ Commented Jan 27, 2018 at 11:49
  • $\begingroup$ @MBaz thanks, I wanted to talk about the case that the rate is maximized over all possible input distributions and all possible encoder-decoder pairs. $\endgroup$ Commented Jan 27, 2018 at 11:50
  • $\begingroup$ @SabrinaCantu If the distribution is not normal, then the maximum rate is less than $C$. $\endgroup$
    – MBaz
    Commented Jan 27, 2018 at 17:13

1 Answer 1

1
$\begingroup$

Wikipedia gives the more general definition. Channel capacity is defined using Information Theory. If you are unfamiliar with that, it will be hard to explain. The channel capacity is always the same for a fixed communications channel (meaning the statistics are fixed).

The more general definition of the capacity $C$ is $$ C = \sup_{p_X(x)} \mathbf{I}(\mathbf{X};\mathbf{Y}) $$ where $\mathbf{I}$ denotes the Mutual Information, $\mathbf{X}$ is the input to the channel and $\mathbf{Y}$ is the output of the channel, and the supremum is taken over all possible input distributions $p_X(x)$.

The equation that you have provided is specific to the AWGN channel. The plots are demonstrating the performance ceiling that exists because you have chosen not to use a Gaussian input sequence (for convenience usually). Notice that it follows the capacity curve to a certain limit but is then limited by the constellation size.

Codewords are relevant when trying to reach capacity, but not generally applicable in the statement of what capacity is. In other words, the channel capacity is independent of the codewords chosen.

$\endgroup$
2
  • $\begingroup$ thanks. As your equation has stated, the capacity is taken by maximization over all possible input distribution. Specifically, in AWGN channel, the capacity is given by the equation is my question by using Gaussian inputs. My question is in the another senses, if I do not take Gaussian inputs, can I be sure that whatever I do, I never reach the channel capacity ? The proof in the original paper of Shannon does not follow this direction (if I understand it correctly). $\endgroup$ Commented Jan 27, 2018 at 11:43
  • $\begingroup$ Maybe I'm not understanding the question exactly then. Perhaps you could expand your text and try to explain better. As it is currently stated, it seems that you are asking if the input distribution chosen changes the capacity. It clearly does not affect the capacity because of the definition of capacity. Are you asking for a quantification of how much "loss" in capacity you get by altering the input distribution? $\endgroup$
    – hops
    Commented Jan 27, 2018 at 21:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.