Maximum Likelihood Through a Noisy Channel

Question

I have random variables $X_1, X_2, \cdots, X_m $, which can take $n$ values and is distributed iid according to $\Theta=(\theta_1, \theta_2, \cdots, \theta_n)$. That is $X_k$ can take values $\{1,2,\cdots,n\}$ and $P(X_k=i)=\theta_i, \ 1\leq k \leq m, \ 1 \leq i \leq n$.

I observe a random variables $Y_1, Y_2, \cdots, Y_m$, where I have that $P(Y_k=j|X_k=i)=c_{j,i}$.

Let $C$ be the matrix $[C_{j,i}]_{1\leq j,i \leq n}$. I have that $C$ is a full rank matrix.

I want to get an ML estimate of $\Theta$, from observations $Y_1, Y_2, \cdots, Y_m$.

Let $\Psi=(\psi_1,\cdots, \psi_n)$, where ${\psi}_i=P(Y_k=i)$. The ML estimate of $\Psi$ is clearly the vector of empirical frequencies. That is, we have the ML estimate of $\psi_i$, is $\hat{\psi}_i = \sum_{\ell=1}^{k} I(Y_{\ell}=i)$, where $I$ is an indicator function. Let us call this $\hat{\Psi}$.

I wanted to know if the ML estimate of $\Theta$, $\hat{\Theta}= C^{-1}\hat{\Psi}$. I can prove it if $n=2$, but do not seem to be able to show this for a general $n$.

Another way to put this is that I have a discrete memory-less source, which has a support of $\{ 1,2, \cdots, n\}$. I can observe this though a discrete memoryless channel, with transition probability matrix $C$. I want to estimate the source symbol probabilities. An ML estimate the observed symbol probabilities can be easily derived to be the empirical frequencies. I wanted to know if it can be shown that the ML estimate of the source symbol probabilities is $C^{-1}$ times the ML estimate of the observed symbol probabilities.

Answer 1

I'm inclined to think this is true, but so far I've only gotten a counterexample:

Consider the channel:

\begin{bmatrix}0.5 & 0.5 & 0 & 0\\ 0 & 0.5 & 0.5 & 0 \\ 0 & 0 & 0.5 & 0.5 \\ 0.5 & 0 & 0 & 0.5\end{bmatrix}

This a typewriter channel. This matrix is not invertible (it has zero determinant, and rank 2). So the ML estimate of $\Theta$ cannot be gotten from this channel.

However, for $p \neq 0.5$ I think it could well be true.

EDIT: In fact I think the noisy channel aspect of this may well be a red-herring. The ML estimator of $\Theta$ is as you describe, simply because you have a linear model. You don't need any conditions of $C$ other than it should be invertible (i.e. all rows are linearly independent).