14 votes

Capacity of AWGN channel

Assuming a channel whose input at each time is a continuous random variable $X$ and its output is $Y=X+Z$, where $Z\sim\mathcal{N}(0,N)$ and $Z$ is independent of $X$, then $$C_{\text{CI-AWGN}}=\frac{...
msm's user avatar
  • 4,255
9 votes
Accepted

Entropy : do we prefer higher or lower entropy?

You seem to have a number of misunderstandings, which I'll try to clarify while also trying to help with your questions. The entropy of a source $H(S)$ gives the average codeword length to encode a ...
MBaz's user avatar
  • 15.3k
7 votes

Capacity of AWGN channel

The capacity formula $$C = 0.5 \log (1+\frac{S}{N}) \tag{1}$$ is for discrete time channel. Assuming you have a sequence of data $\left\lbrace a_n \right\rbrace$ to send out, you need an orthonormal ...
AlexTP's user avatar
  • 6,585
6 votes
Accepted

What is the meaning of channel capacity?

1) Is there a connection between the modulation kind and the channel capacity? The capacity of a channel indicates the upper limit of how many bits can be transmitted per second over the channel with ...
Jim Clay's user avatar
  • 12.1k
6 votes

What is the meaning of channel capacity?

1) Is there a connection between the modulation kind and the channel capacity? Channel capacity is usually defined as the number of information (usually measured by the number of bits) can be sent ...
AlexTP's user avatar
  • 6,585
6 votes
Accepted

Nyquist noiseless channel capacity; how can bit-rate be two times the bandwidth?

I think you're confusing two different (but related) terms. Nyquist says that in a channel of bandwidth $B$ you can transmit up to $2B$ orthogonal pulses per second. So, $R_p \leq 2B$, where $R_p$ is ...
MBaz's user avatar
  • 15.3k
4 votes
Accepted

What is the theoretical probability of error for OOK transmission?

It is not clear whether you want a derivation of the formula for the error probability of a wireless OOK system, or ideas about how the formula might apply to an optical communication system. For a ...
Dilip Sarwate's user avatar
4 votes

How to reconcile "bandwidth" in the Shannon-Hartley Theorem with a spread-spectrum principle

If I'm only using bandwidth B1, doesn't that mean I can filter some of the noise out? And if so, would I be able to get S/N down? Wouldn't that be an alternative to occupying the full channel ...
Marcus Müller's user avatar
4 votes
Accepted

Shannon capacity limit & FEC comparisons

I start with BPSK achieving 1 bits/sec/Hz over passband AWGN. Factoring in 1/3 rate coding this becomes 0.333 bits/sec/Hz. This is not correct. The Shannon noisy channel coding theorem states that ...
AlexTP's user avatar
  • 6,585
4 votes
Accepted

What is the intuition explaining the Shannon-Hartley theorem?

Fundamentally, the signal-to-noise-ratio (SNR or $S/N$) is the ratio of signal power to noise power and that power ratio and is usually expressed in $dB$. Shannon and Hartley (likely collaborating at ...
robert bristow-johnson's user avatar
4 votes

What is the intuition explaining the Shannon-Hartley theorem?

Echoing what already answered: you are approaching this backwards. SNR is a concept that's very fundamental and applicable to way more things that just channel capacity. If you have a signal $y(t)$ ...
Hilmar's user avatar
  • 44k
3 votes
Accepted

Why is bandwidth always limited in a real (physical) channel?

Information bandwidth is dependent on signal to noise ratios. At absolute zero, quantum level signal quantization and quantum noise will limit the lower bound on the noise floor. At higher ...
hotpaw2's user avatar
  • 35.3k
3 votes
Accepted

What is the effect of noise on Shannon entropy?

The grayscale image is actually a discrete image. entropy calculates a histogram and from that extracts "empiric" probabilities, to be used in the common ...
Marcus Müller's user avatar
3 votes

Capacity of AWGN channel

To say that the input signal has a Gaussian distribution means that it is distributed as a Gaussian random variable. In practice, one relies on coding over multiple instances of the channel (in time) ...
hops's user avatar
  • 1,422
3 votes
Accepted

What is the overall capacity of two cascaded BSC channel?

No, this is not correct. Consider the chain of two BSCs with error probabilities $p_1$, $p_2$ as a single BSC with unknown error probability $p$. Now, we know that in overall no error occurs, when: ...
Maximilian Matthé's user avatar
3 votes

Is there a proof that equal bandwidths have equal information-carrying capacity?

I'm not a mathematician so I won't pretend to claim that this is anything like a proof, but at an intuitive level I think that the fact that you can deterministically transform baseband to passband ...
Jim Clay's user avatar
  • 12.1k
3 votes
Accepted

Channel Capacity - is this only for discrete/digital signals?

The definition of channel capacity can be applied to either digital or analog cases. The meaning depends on how you calculate it. The definition is: $$C=\max_{p_X(x)}\ I(X;Y)$$ In the digital world, ...
Tendero's user avatar
  • 5,010
3 votes
Accepted

How to reconcile "bandwidth" in the Shannon-Hartley Theorem with a spread-spectrum principle

You are mixing up two different notions that have little to do with each other. The use of spread-spectrum signaling is not in an effort to achieve (or even approach) the capacity of the (wideband) ...
Dilip Sarwate's user avatar
3 votes
Accepted

Why $H(A)=H(C)$ where $C$ is $A$ with an additional parity bit?

Because the entropy represents information quantity, or if being measured in bit, the smallest number of bits per symbol we need to represent a source. The source $A$ contains $4$ equiprobable ...
AlexTP's user avatar
  • 6,585
3 votes
Accepted

value of 0 log0 in entropy formula

Two possible answers: If a symbol has probability zero, then it does not influence the calculation and there is no need to include it, so you never actually calculate $0\log(0)$. If you insist in ...
MBaz's user avatar
  • 15.3k
2 votes
Accepted

Mutual information of $ \infty $-PAM

I got an answer here, https://math.stackexchange.com/questions/1554659/mutual-information-for-a-continuous-uniform-distribution The $ \text{erf}(x) $ function approximates to 1 with an error less ...
SNR's user avatar
  • 171
2 votes

Can channel capcity be explained in simple terms?

There is a limit where simplifying a scientific concept starts becoming inaccurate. However, the capacity of a channel is a really simple idea that is completely illustrated by the following ...
user304584's user avatar
2 votes
Accepted

Entropy of a dynamical system and source code length

I can try to answer your first question (but I'm not sure I understand exactly what you mean). A source produces messages with average information $H$. You seem to have a binary source, with two ...
MBaz's user avatar
  • 15.3k
2 votes
Accepted

Why are some LDPC codes more immune to noise than others?

As you mention, the LDPC code is completely determined by its generator matrix H. Hence, the properties of H define the (theoretic) performance of the code. Since the LDPC code is a linear code, the ...
Maximilian Matthé's user avatar
2 votes

Given a noise-free channel with limited bandwidth, why is the channel capacity infinite?

Why is this intuitively true? It's a purely hypothetical mathematical formula result. So it doesn't even need to be intuitive, at all. It's very normal for humans to look for intuition in abstract ...
Marcus Müller's user avatar
2 votes

coding gain and shaping gain in SCMA

You can read about shaping gain in any book about digital communication. For example, John G. Proakis, "Digital Communications". The theory is presented in J. H. Conway and N. J. A. Sloane, "Sphere ...
Vyacheslav Klimentyev's user avatar
2 votes

Why is mutual information symmetric but conditional entropy isn't?

There's a cool Venn diagram from here. This shows clearly that $I(X:Y)$ is independent of the order of $X$ and $Y$.
Hugh Perkins's user avatar
2 votes

Will the capacity of a channel becomes unbounded if i increase its signal-to-noise ratio $S/N$ without limit?

By the sampling theorem, uniform samples taken at more than twice the bandwidth of a band-limited signal can be used to reconstruct the signal perfectly by sinc interpolation. For each different set ...
Olli Niemitalo's user avatar
2 votes

How to reconcile "bandwidth" in the Shannon-Hartley Theorem with a spread-spectrum principle

If your wireless channel has bandwidth $B$ but your signal has bandwidth $B_1 < B$, there are a few things you can do: If your data rate is satisfactory, don't use a channel of bandwidth $B$, use ...
MBaz's user avatar
  • 15.3k
2 votes
Accepted

Choosing samples per symbol for modem pulse

The pulse rate is ultimately limited by the channel bandwidth. Assuming you're using raised cosine pulses with rolloff factor $\beta$, then $$B = \frac{(1+\beta)R_p}{2},$$ where $R_p$ is the pulse ...
MBaz's user avatar
  • 15.3k

Only top scored, non community-wiki answers of a minimum length are eligible