The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [information-theory]

The tag has no usage guidance.

16 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
1answer
207 views

Conceptual problem : numberof symbols for nonuniform distribution using entropy : how to determine block size?

A sequence of data of length $N$ can be subdivided into equal sixed blocks each of length (size) $l$. For each block, $w$, we can calculate the entropy known as the block entropy. Considering, entropy ...
2
votes
0answers
53 views

Is there a theoretical limit for how much information hiding can be done through watermarking?

There is a classic channel-capacity notion that exist based on the work of Shannon that gives gives measure of amount of information that can be passed through the channel given its properties. In ...
1
vote
0answers
21 views

Does Mutual information utilize spatial information to calculate image similarity?

To calculate images similarity based on Mutual Information (MI), histogram and join histogram is required. In the literature, there are 2 contradict opinions about MI operation. 1. MI does not utilize ...
1
vote
0answers
38 views

Capacity region of 'symbol asynchronous' Gaussian multiple access channel

I was interested to learn what theoretical limits have been determined so far for the 'multiple access channel' with additive Gaussian noise. This brought me to the 1989 paper by Sergio Verdu, named "...
1
vote
0answers
180 views

SNR for channel after equalization

Suppose I have a channel described by $h[n]$ or $H(z)$ over which I send a simple PAM2 signal. After the channel AWGN is added. What is the best possible SNR with equalization? As suggested in the ...
1
vote
0answers
89 views

Intuitive interpretation of transform coding

I am currently studying source coding of waveform in a course of Information Theory and Coding. More precisely, I am trying to understand, both mathematically and intuitively, transform coding and its ...
1
vote
0answers
610 views

Is there any formula for coding gain of Turbo code?

I want to compare throughput under different MCS( modulation and coding scheme). We know Shannon capacity $C = B \log(1+SNR)$ can be used to calculate throughput by SNR. However, there is no MCS ...
1
vote
0answers
376 views

Extracting features from a signal: a 21st century approach?

I have a discrete signal of random length (let's say between 500 and 800 in time). This signal has an important information which I want to collect and to put it together in a feature vector to train ...
1
vote
0answers
96 views

transmitting data using phase modulation of two sine waves passed through distortion

I have two sine signals with frequencies f1 and f2, and phase p1 and p2. I modulate the phase of these signals to transmit two symbols simultaneously, chosen from the set [0,1,2,3]. For example, I ...
0
votes
0answers
29 views

Capacity achieving codes for BPSK at low SNR

I don't know a lot about comm system design. I have an AWGN channel and low SNR (-3dB). Bandwidth is constrained to some maximum, B. I would like to know good choices of modulation and coding to ...
0
votes
1answer
71 views

How do I measure the amount of information within an image?

I'm trying to find some information threshold that is required for performing detection of objects in images. However, I'm not sure how exactly to quantify the amount of information contained within ...
0
votes
1answer
39 views

Proof that DFT does not require more than N points

I'm trying to show how the discrete Fourier transform (DFT) arises from the equation for the continuous-time Fourier Transform. I've run into an interesting caveat which I can't seem to find an ...
0
votes
0answers
33 views

Symmetric Capacity of a Binary Discrete Memoryless Channel

While reading this paper https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5075875, I found that the the symmetric capacity of BDMC $W$ is given by $$I(W) = \sum_{y\in Y}\sum_{x\in X}\...
0
votes
0answers
19 views

Punctured Turbo Code Decoding: Convergence Issue

I have been trying to create a basic BCJR decoder for a rate (1/2) punctured turbo code. The puncturing is simple, the encoder alternatively selects between the parity bits generated by the two ...
0
votes
0answers
23 views

Two inputs channel capacity

Assume that we have an AWGN channel with two inputs (A,B) and the o/p is written as Y = A + g B + Z where Z is an independent Gaussian noise with zero mean and unit variance and g is a constant ...
0
votes
0answers
351 views

Differences between $\tt wentropy$ in MATLAB and entropy definition

MATLAB offer's a Shannon entropy calculation in its wavelet package: $$ H(X) = -\sum_{i=0}^{N-1} x_i^2\log(x_i^2) $$ I'm wondering what this formula means, since it has no relation with the ...