1
$\begingroup$

The units of channel capacity is bits/second. Does this mean this only refers to discrete/digital signals? Is channel capacity analogous to bandwidth in case of an analog signal transmission?

$\endgroup$
  • $\begingroup$ I am confused. I do understand the link between channel capacity and mutual information, but to not forget that channel capacity, as in the Shannon coding theorem, is a rate of information, expressed in bits/s, while mutual information is expressed in bits. So how can you equate an information rate with an amount of information? $\endgroup$ – edmond Jun 3 at 19:08
3
$\begingroup$

The definition of channel capacity can be applied to either digital or analog cases. The meaning depends on how you calculate it. The definition is:

$$C=\max_{p_X(x)}\ I(X;Y)$$

In the digital world, $p_X(x)$ is a mass probability function and the mutual information is calculated as $H(Y)-H(Y|X)$, where $H(\cdot)$ denotes the entropy.

In the analog case, then $p_X(x)$ is a probability density function and the mutual information is calculated as $h(Y)-h(Y|X)$, where $h(\cdot)$ denotes the differential entropy.

Regarding your other question, channel capacity and bandwidth are not the same thing. Take for example a Gaussian channel, where $$C=B\log_2(1+\mathrm{SNR})$$

You can see that the capacity depends on the bandwidth, but they are clearly not the same.

$\endgroup$
  • 3
    $\begingroup$ To add to this good answer: capacity applies only to digital communication signals; in other words, there is no equivalent for analog AM or FM signals. Having said that, in digital communications you can have a model where the received signal is continuous (for example the AWGN channel) or one where it's discrete (for example the binary symmetric channel), and in each case you get a different expression for the capacity. $\endgroup$ – MBaz Mar 9 '18 at 14:52
  • $\begingroup$ @MBaz There's some entropy in my assessment of your statement: Capacity is the maximum expected mutual information given a channel, $C= \max\limits_{p(x)}I(X;Y)$right? Why would that not apply to the differential entropy case, where $I(X;Y) = D(\,f(x,y)||f(x)f(y)\,)$, thus $$\begin{align}C_{diff.} &= \max\limits_{f(x)} D(\,f(x,y)||f(x)f(y)\,)\\&=\max\limits_{f(x)} \int{f(x)\log\frac fg}(x)\,dx\text,\end{align}$$if I'm not mistaken? Esp. the same $I(X;Y) = h(X) - h(X|Y)$ mutual info/conditional entropy relationship applies,so given the entropy of a noisy reception $h(X|Y)$,I'd see a "capacity". $\endgroup$ – Marcus Müller Mar 9 '18 at 17:48
  • $\begingroup$ @MarcusMüller If I understand well, as soon as you are able to provide a finite value of entropy $h(.)$, this can be considered digital signal. In FM/AM analog message, this measurable information is infinity. And MBaz did not say that the capacity notion can not be applied to differential entropy case (he did mention AWGN channel where this notion is used). $\endgroup$ – AlexTP Mar 9 '18 at 18:19
  • $\begingroup$ @AlexTP hm, I'd define diff entropy $h(X) := -\int f(x)\log f(x)\,dx$, and that would be finite for a lot of non-discrete signals, for example, the common result is that the entropy-maximizing finite support density $f$ is the gaussian, with $h(X\sim\mathcal N(\cdot;\sigma^2))= \frac12\log_2 2\pi e\sigma^2$, which is pretty finite, but doesn't enable me to consider the signal digital? $\endgroup$ – Marcus Müller Mar 9 '18 at 18:27
  • 1
    $\begingroup$ @Eliza There is a difference between discrete/analog data and discrete/analog signals. Both analog and digital information can be encoded as either analog or digital signals. Capacity as defined in my answer only applies to digital signals, but that can correspond to either discrete or analog information (the two different cases explained in the answer). If you can access to a copy of Stallings' Data and Computer Communications, check out the chapter "Signal Encoding Techniques". $\endgroup$ – Tendero Mar 10 '18 at 14:45
0
$\begingroup$

In the audio world, we have this notion of the Gerzon-Craven limit of bit reduction in quantization (this is not the same as lossy coding in a codec like MP3). And it just says that the Shannon result goes both ways (digital info over an analog channel and analog signal over a bit-limited digital channel). And they generalized the Shannon Channel Capacity equation:

$$ C = B \log_2 \left(1+\tfrac{S}{N} \right) $$

to this integral:

$$ C = \int\limits_{0}^{B} \log_2 \left(1+\frac{S(f)}{N(f)} \right) \, \mathrm{d}f $$

$B$ is bandwidth in Hz, $S(f)$ is the power spectrum of the signal, $N(f)$ is the power spectrum of the channel noise (which is, in the reverse direction, the noise from quantization error), $C$ is channel capacity in bits/sec.

So, if you had CD quality audio, that had 96 dB S/N over all of the 22.05 kHz bandwidth, the information needed to represent that 96 dB, 22 kHz audio comes out to be 44100 sample/sec × 16 bits/sample = 705600 bit/sec .

but if you don't need 96 dB S/N over the entire range of human hearing (because of the Fletcher-Munson curve at 0 dB), then you don't need as many bits if you can steer the noise from the quantization error from where your hearing is most sensitive to where it is less sensitive. I dunno what the bit rate comes out to be, but someone can use the integral above (and an expression of the 0 dB curve for $N(f)$) to figure out a smaller bit rate. Of course, good codecs take advantage of other perceptual masking properties of our hearing to reduce the bit rate much further. (I am currently contract working for a company that has a 85000 bit/sec high-quality stereo codec.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.