# Questions tagged [information-theory]

The tag has no usage guidance.

103 questions
Filter by
Sorted by
Tagged with
184 views

### Shannon capacity limit & FEC comparisons

In this figure from Fundamentals of Digital Communications by Madhow , pp 328 there is a comparison of a rate 1/3 turbo code with the shannon limit for rate 1/3 BPSK: Trying to understand the limit ...
104 views

### Soft-output decoder

I am trying to implement the soft-output for my soft-decoder, however, the resulting log-likelihood ratios are not corresponding to the corrected hard bits. The soft-output regards the LLR-values for ...
165 views

### value of 0 log0 in entropy formula

Why is the value of $p_i\log(p_i)$ in entropy formula considered $0$ when $p_i =0$? I get that it is the limiting value, but does that mean $p$ is never equals to zero, but only tends to?
26 views

### Is there a way to describe the pattern of a signal as a numeric value?

I've been digging a little into signal processing and I just started a networks and telecommunications class at college. We saw there are different encode techniques Amplitude, Frequency and Phase ...
49 views

### Secret Sharing & Information Theory

I'm reading a paper -- Common Randomness in Information Theory and Cryptograph-secret. And I do have trouble in understanding formula (2.6): $$\frac {1}{n} H(K)> H-\epsilon \tag{2.6}\label{2.6},$$...
230 views

33 views

### Why the requirement of the GCD of the lengths of all circuits in the graph being one?

I am reading A Mathematical Theory of Communication. The second requirement of an ergodic process confuses me (emphasis mine): All the examples of artificial languages given above are ergodic. This ...
26 views

375 views

### How to calculate the entropy of random normally distributed samples?

I have a hardware device that outputs a normal distribution of readings. It's (fitted) parameters are mean = 35 units, std.dev. = 8 units and I read the values as 8 bit integers. I'm trying to ...
310 views

### coding gain and shaping gain in SCMA

It is said that sparse code multiple access (SCMA) has the advantage of higher shaping and coding gain over LDS-CDMA due to the multidimensional constellation. What exactly is the shaping gain and how ...
329 views

### SNR for channel after equalization

Suppose I have a channel described by $h[n]$ or $H(z)$ over which I send a simple PAM2 signal. After the channel AWGN is added. What is the best possible SNR with equalization? As suggested in the ...
57 views

### Is there a theoretical limit for how much information hiding can be done through watermarking?

There is a classic channel-capacity notion that exist based on the work of Shannon that gives gives measure of amount of information that can be passed through the channel given its properties. In ...
114 views

### Intuitive interpretation of transform coding

I am currently studying source coding of waveform in a course of Information Theory and Coding. More precisely, I am trying to understand, both mathematically and intuitively, transform coding and its ...
I am confused understanding basic concepts of communication over AWGN channels. I know the capacity of a discrete time AWGN channel is: $$C=\frac{1}{2}\log_2\left(1+\frac{S}{N}\right)$$ and it is ...