# How to calculate the following mutual infomation between $X$ and $Y$ in matlab or python?

Recently I meet the following question: Let $$X\in {\mu_1,\mu_2,\cdots,\mu_m}$$, $$Y \in {R_1,R_2,\cdots,R_n}$$, where $$\mu_i, i = 1,2,\cdots,m$$ is means of $$m$$ gauss probablity density function, $$Y$$ is the output of a discrete memoryless channel (DMC), probabilities $$p_{ij} = P(Y=R_j|X=\mu_i)$$ denote the crossover probabilities, Assuming $$X$$ is a uniform random variable, then for $$m = 4, n = 7$$, the mutual infomation between $$X$$ and $$Y$$ is stated as follows: $$I(X;Y)=H(Y)-H(Y|X)=H(\frac{\sum_{i=1}^4 p_{i1}}{4}+\frac{\sum_{i=1}^4 p_{i2}}{4}+\cdots +\frac{\sum_{i=1}^4 p_{i7}}{4})- \frac{1}{4}[H(p_{11},p_{12},\cdots,p_{17})+H(p_{21},p_{22},\cdots,p_{27})+\cdots+H(p_{71},p_{72},\cdots,p_{77})]$$ here $$H(\cdot)$$ is the entropy function, so the question is:

How to get the crossover probabilities $$p_{ij}$$ and how to calculate the maximal of $$I(X;Y)$$ and otuput the optimal $$R_j$$? Besides, how to handle the general case? i.e. $$\forall m, n > 0 \in \mathbb{Z}$$? The original paper's name is Optimization of Read Thresholds in NAND Flash Memory for LDPC Codes". Any advice or anser is very helpful, thank you. I've tried to send an e-mail to the author, but with no reply.

Update: Under what conditions can we get the crossover probabilities $$p_{ij}$$ and calculate the maximum of $$I(X;Y)$$ which gives out the optimal $$R_j$$?

Update Again: Follows Mr.@Max's suggestion, I've wrote a matlab file, which is bellow:

clear all
close all
clc

m = 6;
N = 4;
delta_g = 0.5;
epsilon = 1e-3;
zeta    = 1e-8;
gamma_vect = [-2.4,-1.6,-0.4,0.4,1.6,2.4];
N_0   = 0.1;
it    = 0;
mu1   = -3;
mu2   = -1;
mu3   = 1;
mu4   = 4;
sigma1 = 1;
sigma2 = 1;
sigma3 = 1;
sigma4 = 1;
f_11 = @(x)sqrt(2*pi*sigma1^2).*exp(-(x-mu1).^2./(2*sigma1^2));
f_10 = @(x)sqrt(2*pi*sigma2^2).*exp(-(x-mu2).^2./(2*sigma2^2));
f_00 = @(x)sqrt(2*pi*sigma3^2).*exp(-(x-mu3).^2./(2*sigma3^2));
f_01 = @(x)sqrt(2*pi*sigma4^2).*exp(-(x-mu4).^2./(2*sigma4^2));
prob_tran = zeros(4, 7);
prob_tran(1,1) = integral(f_11, -Inf, gamma_vect(1));
prob_tran(1,2) = integral(f_11, gamma_vect(1),+Inf);
prob_tran(2,1) = integral(f_11, -Inf, gamma_vect(2));
prob_tran(2,2) = integral(f_11, gamma_vect(2), gamma_vect(3));
prob_tran(2,3) = integral(f_11, gamma_vect(3), +Inf);
prob_tran(3,1) = integral(f_10, -Inf, gamma_vect(4));
prob_tran(3,2) = integral(f_10, gamma_vect(4),gamma_vect(5));
prob_tran(3,3) = integral(f_10, gamma_vect(5),gamma_vect(6));
prob_tran(4,6) = integral(f_10, -Inf,gamma_vect(6));
prob_tran(4,7) = integral(f_10, gamma_vect(6), +Inf);


and the probablity matrix is:

$$4.5600\ 1.7232\ 0\ 0\ 0\ 0\ 0$$ $$5.7758\ 0.4781\ 0.0293\ 0\ 0\ 0\ 0$$ $$5.7758\ 0.4781\ 0.0272\ 0\ 0\ 0\ 0$$ $$0\ 0\ 0\ 0\ 0\ 6.2811\ 0.0021$$ which seems wrong, another question is how to calculate the entropy function $$H(Y)$$ and $$H(Y|X)$$ using $$p_{ij}$$?

• It's unsymmetric because $\mu_4$ is off by 1. And a normalization seems to be missing. The summation of your probabilities is always $2\pi$. Otherwise, that does seem plausible.
– Max
Sep 6, 2023 at 6:07
• @Max, dear Max, another question, how to calculate $H(Y)$ and $H(Y|X)$ via $p_{ij}$? Sep 6, 2023 at 6:34

• Thank you, sir. If we have the infomation of $f_{11},f_{10},etc$, could we derive the transition probabilities? Sep 5, 2023 at 11:08
• Well, yes. Just look at the figure. For $f_{11}$ for example, you would integrate from $-\infty$ to $\gamma_1$ to get $p_{11}$, from $\gamma_1$ to $\gamma_2$ to get $p_{12}$ and so on.