3
$\begingroup$

Suppose I would like to stream data from a device in a single second. What would be the theoretical upperbound of the amount of information (in bits, bytes, pixels...) that could be sent during this period?

What determines this upper bound?

$\endgroup$
  • $\begingroup$ as indicated by @MBaz, signal-to-noise ratio and bandwidth is what determines this upper bound. $\endgroup$ – robert bristow-johnson Mar 26 '15 at 4:33
  • $\begingroup$ Theoretical upperbound in math would be infinite in zero noise. Theoretical upperbound in physics would be bounded by quantum mechanics and gravity (Planck dimension quantization noise compared to the max possible energy density before the device became a black hole). $\endgroup$ – hotpaw2 Mar 26 '15 at 19:11
5
$\begingroup$

There are many factors involved in understanding the theoretical limits to communication. What follows is just a brief introduction that only scratches the surface.

First, let's consider a simple scenario: there is no noise and no distortion of the signal being transmitted. We do allow for attenuation. Under these circumstances, you can transmit up to $R_p=2B$ pulses per second. $B$ is the bandwidth you have available. For example: if you have 10 kHz of bandwidth, you may transmit up to 20,000 pulses in one second. $R_p$ is called the Nyquist rate, after Harry Nyquist.

However, your question is about bits, not pulses. It turns out that you can transmit bits using pulses. Let's say you want to transmit the bit sequence $1001$. You could do this by transmitting pulses with amplitudes $+1, -1, -1, +1$. (The pulse shape does not matter, but they do need to "fit" in the bandwidth you have available.) If you do this, your bit rate $R_b$ is equal to your pulse rate, $R_b=R_p$.

You can transmit more than one bit per pulse, though. Consider allowing the pulses to have four amplitudes $+3,+1,-1,-3$. Then you could transmit the sequence $1001$ with only two pulses, of amplitudes, say, $+1,-3$. You're free to map bits to pulse amplitudes in any way you want. In this scenario, you have $R_b=2R_p$.

So, in theory, if there is no noise, then the pulse rate is limited to twice your bandwith, but the bit rate can be as large as you want, by allowing the pulses to take more and more different amplitudes.

Now consider a more realistic scenario, where noise is present. Now, the received signal is not just what you transmitted, but it includes noise too. Assume that you're transmitting with power equal to $S$ watts, and that the noise over your bandwidth $B$ has power $N$ watts. Now your bit rate is no longer as large as you want; it cannot be larger than

$$C=B\log_2(1+S/N).$$

$C$ is known as Shannon's channel capacity, after Claude Shannon. However, keep in mind that transmitting at, or near, capacity may involve a very complex coding system. In practice, a simple system will operate well below the channel capacity. Even very complex systems may not achieve the capacity.

What happens if you only have a simple system, and you attempt to communicate at a significant fraction of the capacity? In that case, you're likely to have errors in your communication. An error happens when the receiver mistakes a bit 1 for a bit 0 or vice versa. Depending on your application, a certain amount of errors can be tolerated. For example, in voice communications one error out of 1000 bits (a bit error rate (BER) of 0.001) is acceptable. Other systems are intolerant to errors: for example, you don't want your hard drive to corrupt your files, not even once a year.

In more complex situations the capacity may be much smaller and the systems that achieve high bit rates become much more complex. For example, in wireless communications the transmitted signal suffers not only from noise, but also from echos and Doppler shift.

In summary, as you see, there is no simple answer. Let me know if you need clarification, or more details.

$\endgroup$
  • 2
    $\begingroup$ M, both $S$ and $N$ are considered to have the same flat power spectrum over the bandwidth of width $B$ in the capacity formula above. to generalize it, if $S(f)$ is the power spectrum of the signal and $N(f)$ is the power spectrum of the noise, then the channel capacity (in bits per unit time) is $$ C = \int\limits_{0}^{B} \log_2\left(1 + \frac{S(f)}{N(f)} \right) df $$ $\endgroup$ – robert bristow-johnson Mar 26 '15 at 4:37
  • $\begingroup$ Is this answer assuming digital signal transmission over copper? Let's say by contrast we used a single fiber optic cable with good shielding (low noise) to encode digital information into visible light; 255 different intensities per frequency, and 1nm wavelength divisions, with a prism at the other end to separate them out for decoding. Is such a system limited by the same mechanisms and formulas? $\endgroup$ – Ehryk Mar 26 '15 at 5:28
  • $\begingroup$ Or, for that matter, whatever that limit is we just add 1 or 10 or 100 or 10,000 (or n) additional fiber optic / copper channels. While this isn't very reasonable, 'device' is vague enough to allow n connections, so you could eventually get to any bit rate you wanted just by adding more channels (unless it's wireless). $\endgroup$ – Ehryk Mar 26 '15 at 5:30
  • $\begingroup$ @Ehryk A single mode fiber has the same capacity as given in the answer as long as its nonlinearity is negligible. In reality, nonlinear effects have to be taken into account and the capacity of the nonlinear fiber channel is an ongoing research topic. Note that the dominant noise sources in optical communications are the optical amplifiers and the electric receiver circuit. If you add spatial multiplex as suggested in your second comment the capacity multiplies by N (assuming that the spatial channels do not interfere with each other). $\endgroup$ – Deve Mar 26 '15 at 8:24
  • 1
    $\begingroup$ @Ehryk The general concept of capacity (a maximum achievable bit rate with negligible errors) applies to any communication system; that is, to any system with an information source, a channel, and an information sink. The particular formula for the capacity depends on the channel and in the way you use it. For example, a noisy channel has a different capacity formula than a wireless channel; a wireless channel in which you transmit/receive with many antennas at the same time has yet another formula. A fiber will have another (maybe not yet known, as Deve points out). $\endgroup$ – MBaz Mar 26 '15 at 13:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.