0
$\begingroup$

I am currently studying about Polar Codes in 5G standard and while reading my paper I found something called AEP which is required for channel coding. I surfed the web but didn't found a satisfying answer. Can someone explain what it is, clearly?

$\endgroup$
0
$\begingroup$

When you are sampling a stochastic process $n$ times, the larger you make $n$, the higher the probability that the series of samples is contained in the so called strongly typical set of outcomes of length $n$, where all members have roughly the same probability to be realized. If you let $n\rightarrow \infty$ it holds that $$-\frac{1}{n}\log p(X_1,X_2,...,X_n)\rightarrow H(X)$$ with $H(X)$ being the entropy rate of the process.

This should be intuitively clear: the longer the sequence, the more possible outcomes, each with possiblity near zero but positive. The number of typical outcomes with $p\approx\epsilon$ is growing faster than the number of non typical outcomes as $n$ is growing. As $n\rightarrow \infty$, the "leftover probability" for non typical outcomes gets smaller and smaller, thus making the above equation valid.

$\endgroup$
  • $\begingroup$ What is $\frac{1}{n}$, is it the probability of equiprobable random variables $X_{i}$? And also, if $H(X)$ is the entropy, shouldn't there be a summation over here which would be equivalent of the expectation of the information contained? $\endgroup$ – Himanshu Sharma Apr 9 at 14:29
  • $\begingroup$ I'm sorry, $H(X)$ is the entropy rate, I corrected it in my posting. That's where the $\frac{1}{n}$ is coming from. $\endgroup$ – Max Apr 10 at 5:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.