Questions tagged [information-theory]

The tag has no usage guidance.

18 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
1answer
213 views

Conceptual problem : numberof symbols for nonuniform distribution using entropy : how to determine block size?

A sequence of data of length $N$ can be subdivided into equal sixed blocks each of length (size) $l$. For each block, $w$, we can calculate the entropy known as the block entropy. Considering, entropy ...
2
votes
0answers
54 views

Is there a theoretical limit for how much information hiding can be done through watermarking?

There is a classic channel-capacity notion that exist based on the work of Shannon that gives gives measure of amount of information that can be passed through the channel given its properties. In ...
1
vote
0answers
14 views

Pearson correlation coefficient vs mutual information

I recently computed both the pearson coefficient and the mutual information of two time series as well as the corresponding error bars, and what I find out in the end is a significant level of ...
1
vote
0answers
23 views

Does Mutual information utilize spatial information to calculate image similarity?

To calculate images similarity based on Mutual Information (MI), histogram and join histogram is required. In the literature, there are 2 contradict opinions about MI operation. 1. MI does not utilize ...
1
vote
0answers
40 views

Capacity region of 'symbol asynchronous' Gaussian multiple access channel

I was interested to learn what theoretical limits have been determined so far for the 'multiple access channel' with additive Gaussian noise. This brought me to the 1989 paper by Sergio Verdu, named "...
1
vote
0answers
218 views

SNR for channel after equalization

Suppose I have a channel described by $h[n]$ or $H(z)$ over which I send a simple PAM2 signal. After the channel AWGN is added. What is the best possible SNR with equalization? As suggested in the ...
1
vote
0answers
98 views

Intuitive interpretation of transform coding

I am currently studying source coding of waveform in a course of Information Theory and Coding. More precisely, I am trying to understand, both mathematically and intuitively, transform coding and its ...
1
vote
0answers
633 views

Is there any formula for coding gain of Turbo code?

I want to compare throughput under different MCS( modulation and coding scheme). We know Shannon capacity $C = B \log(1+SNR)$ can be used to calculate throughput by SNR. However, there is no MCS ...
1
vote
0answers
381 views

Extracting features from a signal: a 21st century approach?

I have a discrete signal of random length (let's say between 500 and 800 in time). This signal has an important information which I want to collect and to put it together in a feature vector to train ...
1
vote
0answers
96 views

transmitting data using phase modulation of two sine waves passed through distortion

I have two sine signals with frequencies f1 and f2, and phase p1 and p2. I modulate the phase of these signals to transmit two symbols simultaneously, chosen from the set [0,1,2,3]. For example, I ...
0
votes
0answers
31 views

Information entropy of Poisson noise

I would like to calculate the information entropy of the channel with additive Poisson noise. The model consists of an input signal, an optical image, an image acquisition device, and the output ...
0
votes
0answers
36 views

Capacity achieving codes for BPSK at low SNR

I don't know a lot about comm system design. I have an AWGN channel and low SNR (-3dB). Bandwidth is constrained to some maximum, B. I would like to know good choices of modulation and coding to ...
0
votes
1answer
49 views

Proof that DFT does not require more than N points

I'm trying to show how the discrete Fourier transform (DFT) arises from the equation for the continuous-time Fourier Transform. I've run into an interesting caveat which I can't seem to find an ...
0
votes
0answers
36 views

Symmetric Capacity of a Binary Discrete Memoryless Channel

While reading this paper https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5075875, I found that the the symmetric capacity of BDMC $W$ is given by $$I(W) = \sum_{y\in Y}\sum_{x\in X}\...
0
votes
0answers
20 views

Punctured Turbo Code Decoding: Convergence Issue

I have been trying to create a basic BCJR decoder for a rate (1/2) punctured turbo code. The puncturing is simple, the encoder alternatively selects between the parity bits generated by the two ...
0
votes
0answers
23 views

Two inputs channel capacity

Assume that we have an AWGN channel with two inputs (A,B) and the o/p is written as Y = A + g B + Z where Z is an independent Gaussian noise with zero mean and unit variance and g is a constant ...
0
votes
0answers
364 views

Differences between $\tt wentropy$ in MATLAB and entropy definition

MATLAB offer's a Shannon entropy calculation in its wavelet package: $$ H(X) = -\sum_{i=0}^{N-1} x_i^2\log(x_i^2) $$ I'm wondering what this formula means, since it has no relation with the ...
-1
votes
1answer
41 views

Bandwidth and probability of continuous signal

I'm a little bit confused as to how to calculate the bandwidth of a signal. A question has me believing that it is correlated with the probability distribution. I am however not sure of this. ...