# Questions tagged [information-theory]

The tag has no usage guidance.

110 questions
Filter by
Sorted by
Tagged with
1 vote
62 views

### Lower bound on information or entropy?

Inspired by this question: Does a simple photograph contain more information than a complex painting? Say that I have a discrete representation of some physical object - eg an image file. What is the ...
• 2,612
123 views

### How to code Rician fading channel gains from k-factor?

I know that MATLAB provides some built-in functions but I am not interested in those. I want to know how can we code to obtain the Rician fading channel gains from normal or Gaussian distributions. ...
• 23
185 views

i'm currently working on a circuit transient simulation, and I'm wondering how the type of input excitation can affect the final result. In particular the focus is on impulse input vs step input, to ...
1 vote
459 views

### What is the intuition explaining the Shannon-Hartley theorem?

In the digital domain, the signal to noise ratio (SNR) may be interpreted loosely but usefully as the number of discrete or quantized levels one can transmit reliably. For instance, loosely speaking, ...
• 301
49 views

### What are the differences between GPS and GLONASS?

I'm studying the localization systems, especially the GPS one. I inquired about other systems and wanted to understand what are the main differences between GPS and GLONASS, why has the first type ...
1 vote
89 views

### Entropy Loss in Linear Filters

I've been trying to figure out the result for the entropy loss/entropy gain in linear systems derived in "The Mathematical Theory of Communication" by Claude Shannon. Claude Shannon states ...
36 views

### Proof that block of independent Gaussian symbols has higher mutual information than correlated Gaussian symbols in band- & power-limited AWGN channel

Given a discrete-time AWGN channel defined by: $$Y = X + N$$ where $N$ is a zero-mean Gaussian RV with $E[N^2] = P_N$ representing additive noise, $X$ is the transmitted symbol with power ...
• 2,172
435 views

### Shannon capacity limit & FEC comparisons

In this figure from Fundamentals of Digital Communications by Madhow , pp 328 there is a comparison of a rate 1/3 turbo code with the shannon limit for rate 1/3 BPSK: Trying to understand the limit ...
• 743
1 vote
111 views

### Soft-output decoder

I am trying to implement the soft-output for my soft-decoder, however, the resulting log-likelihood ratios are not corresponding to the corrected hard bits. The soft-output regards the LLR-values for ...
1 vote
1k views

### value of 0 log0 in entropy formula

Why is the value of $p_i\log(p_i)$ in entropy formula considered $0$ when $p_i =0$? I get that it is the limiting value, but does that mean $p$ is never equals to zero, but only tends to?
• 15
33 views

### Is there a way to describe the pattern of a signal as a numeric value?

I've been digging a little into signal processing and I just started a networks and telecommunications class at college. We saw there are different encode techniques Amplitude, Frequency and Phase ...
1 vote
50 views

### Secret Sharing & Information Theory

I'm reading a paper -- Common Randomness in Information Theory and Cryptograph-secret. And I do have trouble in understanding formula (2.6): $$\frac {1}{n} H(K)> H-\epsilon \tag{2.6}\label{2.6},$$...
• 11
456 views

1 vote
50 views

### Why the requirement of the GCD of the lengths of all circuits in the graph being one?

I am reading A Mathematical Theory of Communication. The second requirement of an ergodic process confuses me (emphasis mine): All the examples of artificial languages given above are ergodic. This ...
• 113