Recently I meet the following question: Let $X\in {\mu_1,\mu_2,\cdots,\mu_m}$, $Y \in {R_1,R_2,\cdots,R_n}$, where $\mu_i, i = 1,2,\cdots,m$ is means of $m$ gauss probablity density function, $Y$ is the output of a discrete memoryless channel (DMC), probabilities $p_{ij} = P(Y=R_j|X=\mu_i)$ denote the crossover probabilities, Assuming $X$ is a uniform random variable, then for $m = 4, n = 7$, the mutual infomation between $X$ and $Y$ is stated as follows: $$I(X;Y)=H(Y)-H(Y|X)=H(\frac{\sum_{i=1}^4 p_{i1}}{4}+\frac{\sum_{i=1}^4 p_{i2}}{4}+\cdots +\frac{\sum_{i=1}^4 p_{i7}}{4})- \frac{1}{4}[H(p_{11},p_{12},\cdots,p_{17})+H(p_{21},p_{22},\cdots,p_{27})+\cdots+H(p_{71},p_{72},\cdots,p_{77})]$$ here $H(\cdot)$ is the entropy function, so the question is:

How to get the crossover probabilities $p_{ij}$ and how to calculate the maximal of $I(X;Y)$ and otuput the optimal $R_j$? Besides, how to handle the general case? i.e. $\forall m, n > 0 \in \mathbb{Z}$? Equivalent discrete memoryless channel (DMC) model for MLC The original paper's name is ``Optimization of Read Thresholds in NAND Flash Memory for LDPC Codes". Any advice or anser is very helpful, thank you. I've tried to send an e-mail to the author, but with no reply.

Update: Under what conditions can we get the crossover probabilities $p_{ij}$ and calculate the maximum of $I(X;Y)$ which gives out the optimal $R_j$?

Update Again: Follows Mr.@Max's suggestion, I've wrote a matlab file, which is bellow:

clear all
close all

m = 6;
N = 4;
delta_g = 0.5;
epsilon = 1e-3;
zeta    = 1e-8;
gamma_vect = [-2.4,-1.6,-0.4,0.4,1.6,2.4];
N_0   = 0.1;
it    = 0;
mu1   = -3;
mu2   = -1;
mu3   = 1;
mu4   = 4;
sigma1 = 1;
sigma2 = 1;
sigma3 = 1;
sigma4 = 1;
f_11 = @(x)sqrt(2*pi*sigma1^2).*exp(-(x-mu1).^2./(2*sigma1^2));
f_10 = @(x)sqrt(2*pi*sigma2^2).*exp(-(x-mu2).^2./(2*sigma2^2));
f_00 = @(x)sqrt(2*pi*sigma3^2).*exp(-(x-mu3).^2./(2*sigma3^2));
f_01 = @(x)sqrt(2*pi*sigma4^2).*exp(-(x-mu4).^2./(2*sigma4^2));
prob_tran = zeros(4, 7);
prob_tran(1,1) = integral(f_11, -Inf, gamma_vect(1));
prob_tran(1,2) = integral(f_11, gamma_vect(1),+Inf);
prob_tran(2,1) = integral(f_11, -Inf, gamma_vect(2));
prob_tran(2,2) = integral(f_11, gamma_vect(2), gamma_vect(3));
prob_tran(2,3) = integral(f_11, gamma_vect(3), +Inf);
prob_tran(3,1) = integral(f_10, -Inf, gamma_vect(4));
prob_tran(3,2) = integral(f_10, gamma_vect(4),gamma_vect(5));
prob_tran(3,3) = integral(f_10, gamma_vect(5),gamma_vect(6));
prob_tran(4,6) = integral(f_10, -Inf,gamma_vect(6));
prob_tran(4,7) = integral(f_10, gamma_vect(6), +Inf);

and the probablity matrix is:

$$4.5600\ 1.7232\ 0\ 0\ 0\ 0\ 0$$ $$5.7758\ 0.4781\ 0.0293\ 0\ 0\ 0\ 0$$ $$5.7758\ 0.4781\ 0.0272\ 0\ 0\ 0\ 0$$ $$0\ 0\ 0\ 0\ 0\ 6.2811\ 0.0021$$ which seems wrong, another question is how to calculate the entropy function $H(Y)$ and $H(Y|X)$ using $p_{ij}$?

  • $\begingroup$ It's unsymmetric because $\mu_4$ is off by 1. And a normalization seems to be missing. The summation of your probabilities is always $2\pi$. Otherwise, that does seem plausible. $\endgroup$
    – Max
    Sep 6, 2023 at 6:07
  • $\begingroup$ @Max, dear Max, another question, how to calculate $H(Y)$ and $H(Y|X)$ via $p_{ij}$? $\endgroup$
    – Milin
    Sep 6, 2023 at 6:34

1 Answer 1


Short answer: you can't!

Long answer: Without further information, it is not possible to calculate either the transition probabilities or the mutual information. Besides the fact, that the input is uniformly distributed and the number of inputs and outputs, nothing is provided, the system is not determinded. I looked at the paper you mentioned (which is actually a thesis). The author does not derive these values analytically (which he can't). The DMC is just brought in for clarity reasons, finding the maximum mutual information is done numerically and iteratively coming from these distributions: Picture taken from 1 For this, there is some pseudo code given and the algorithm as well as the results are explained in detail.

If you want to get into some more detail regarding the theoretical background, I would recommend to read this book.

  • $\begingroup$ Thank you, sir. If we have the infomation of $f_{11},f_{10},etc$, could we derive the transition probabilities? $\endgroup$
    – Milin
    Sep 5, 2023 at 11:08
  • $\begingroup$ Well, yes. Just look at the figure. For $f_{11}$ for example, you would integrate from $-\infty$ to $\gamma_1$ to get $p_{11}$, from $\gamma_1$ to $\gamma_2$ to get $p_{12}$ and so on. $\endgroup$
    – Max
    Sep 5, 2023 at 11:38
  • $\begingroup$ thank you for your kind reply! $\endgroup$
    – Milin
    Sep 5, 2023 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.