# Questions tagged [information-theory]

The tag has no usage guidance.

87 questions
Filter by
Sorted by
Tagged with
26 views

### Band-limited signal recovery with finite bit depth

The sampling theorem tells us that a signal with no frequencies above $f$ can be completely described by sampling it a rate of $2f$. However, the theorem makes no reference to quantization, and so I ...
11 views

### Pearson correlation coefficient vs mutual information

I recently computed both the pearson coefficient and the mutual information of two time series as well as the corresponding error bars, and what I find out in the end is a significant level of ...
80 views

### Nyquist noiseless channel capacity; how can bit-rate be two times the bandwidth?

I'm confused by the Nyquist channel capacity formula. How can channel maximum capacity approach double the bandwidth. $C = 2\times BW \times log_{2}(L)$ bits/sec The way it was explained to me ...
33 views

### Capacity achieving codes for BPSK at low SNR

I don't know a lot about comm system design. I have an AWGN channel and low SNR (-3dB). Bandwidth is constrained to some maximum, B. I would like to know good choices of modulation and coding to ...
16 views

### How good is the Leech lattice for AWGN coding or MSE quantization of Gaussian sources?

Anyone know where to find graphs showing how well it works vs other schemes? thanks
122 views

### How do I measure the amount of information within an image?

I'm trying to find some information threshold that is required for performing detection of objects in images. However, I'm not sure how exactly to quantify the amount of information contained within ...
40 views

### Proof that DFT does not require more than N points

I'm trying to show how the discrete Fourier transform (DFT) arises from the equation for the continuous-time Fourier Transform. I've run into an interesting caveat which I can't seem to find an ...
89 views

19 views

### Punctured Turbo Code Decoding: Convergence Issue

I have been trying to create a basic BCJR decoder for a rate (1/2) punctured turbo code. The puncturing is simple, the encoder alternatively selects between the parity bits generated by the two ...
23 views

### Two inputs channel capacity

Assume that we have an AWGN channel with two inputs (A,B) and the o/p is written as Y = A + g B + Z where Z is an independent Gaussian noise with zero mean and unit variance and g is a constant ...
33 views

### Relationship between spectral efficiency and PAR

Low PAR means less fluctiations in the magnitude of a signal and hence analog signal processing (e.g., power amplifier) is tremendously reduced. For example, constant envelope modulation schemes like ...
143 views

### Concatenated Deletion & Sticky Channel Capacity

Suppose you are sending information through a deletion channel with a deletion rate of .5 and then the output of this channel is sent through a sticky channel with a repeat rate of .5 also. No error ...
386 views

### White noise and entropy confusion about concepts

The spectral entropy (SE) of a signal is a measure of its spectral power distribution. The concept is based on the Shannon entropy, or information entropy, in information theory. The SE treats the ...
42 views

72 views

### Why are some LDPC codes more immune to noise than others?

When one analyzes an LDPC code, it is usual to plot the BER versus the $E_b / N_0$. My questions are: Why are some LDPC codes better than others in terms of noise immunity? After all, they are just ...
1k views

### How to explain this BER vs SNR behaviour?

I wrote a script in MATLAB to simulate an LDPC created using an algorithm of a colleague of mine. The parity check-matrix is regular and has dimensions $1012 \times 1518$ and the code rate is $R=1/3$. ...
612 views

### What is the overall capacity of two cascaded BSC channel?

Is it true:$C = C_1 * C_2$ where $C_1$ and $C_2$ are the two cascaded BSC which makes $C$.
354 views

### Differences between $\tt wentropy$ in MATLAB and entropy definition

MATLAB offer's a Shannon entropy calculation in its wavelet package: $$H(X) = -\sum_{i=0}^{N-1} x_i^2\log(x_i^2)$$ I'm wondering what this formula means, since it has no relation with the ...
303 views

### Source and channel encoding

In information theory, there are two concepts that are widely used: source encoding and channel encoding. I'm afraid I may not be capable of expressing my doubts clearly, but the thing is that I don'...
59 views

### Discrete entropies

I've been given a problem where I need to find the entropy of two random variables. I can find part of the answer, but not all of it. I am given the following: $X$ is a uniformly distributed random ...
I am trying to simulate an optical wireless communication channel which uses OOK modulation. Looking at equation 14 in this literature, I found that: For electrical SNR at the receiver \textrm{...