0
$\begingroup$

The received noisy signal $y_n \in \mathbb{R}$ is expressed as: \begin{align} y_n = \mathbf{h}^\mathsf{T}\mathbf{u}_n + w_n. \tag{1} \end{align} $\mathbf{h} = [h_0,h_1,\ldots,h_{p-1}]^\mathsf{T} \in \mathbb{R}^{1 \times p}$ of length $p$ which represents the impulse response of length $p$, and $\mathbf{u}_{n} = [u_{n}, u_{n-1}, u_{n-2},\ldots,u_{n-p+1}]^\mathsf{T}$ having variance $\sigma^2$. $w \sim N(0,\sigma^2_w)$ is the Additive White Gaussian noise. $\mathbf{y} = [y_0,y_1,\ldots,y_N]$ and $\theta = [\mathbf{h},\sigma^2_w]$ I will be estimating the following terms of the model below using $y$. The estimation of the model is formulated as follows: \begin{align} z_n = {\mathbf{h}}^\mathsf{T}{\mathbf{u}_n}. \tag{2} \end{align}

I want to use a variable $z$ to separate what is known and unknown. Therefore, I have used $z$. But, if I use this new variable, the mathematical formulation should have the term $z$.
The condition pdf $\mathbf{y}$ given $\theta$ is given by $P(\mathbf{y}|\mathbf{\theta}) =\prod_{n=1}^{N}\frac{1}{\sqrt{2\pi \sigma^2_w}}\exp(-\frac{(y_n -z_n)(y_n -z_n)^T}{2\sigma^2_w})$

My confusions are:

1) What notations to use for probability density function is it the one below:

$\mathsf{P}_y(y_n|{\mathbf{u}_n})$ or $\mathsf{P}_z(z_n|{\mathbf{u}_n})$ what goes in the subscript if I want to use $z$?

2) Is $P(\mathbf{y}|\mathbf{\theta})$ correct or $P_y(\mathbf{y}|\mathbf{\theta})$ or $P_z(\mathbf{z}|\mathbf{\theta})$?

3) If I want to use expectation maximization, then would $\xi$ which denotes the complete data set consisting of the input signal and the received noisy signal be written as: \begin{align} \xi &= \{u_0,\ldots,u_{N-1},y_0,\ldots,y_{N-1} \} \tag{3} \end{align} or \begin{align} \xi &= \{\hat{u_0},\ldots,\hat{u_{N-1}},z_0,\ldots,z_{N-1} \} \tag{4} \end{align}

4) Would the posterior be written as \begin{align} p_{\theta}(y_{1:N}|u_{1:N})p_{\theta}(u_{1:N}) = \prod_{n=1}^{N} \mathsf{P}_u(u_n|{u}_{n+1}) \prod_{n=1}^{N} \mathsf{P}_y(y_n|\mathbf{u}_n). \tag{5} \end{align} or \begin{align} p_{\theta}(z_{1:N}|u_{1:N})p_{\theta}(u_{1:N}) = \prod_{n=1}^{N} \mathsf{P}_u(u_n|{u}_{n+1}) \prod_{n=1}^{N} \mathsf{P}_z(z_n|\mathbf{u}_n). \tag{6} \end{align} Please help what the correct notations are used to represent the pdf, likelihood, log-likelihood if I use the variable $z$ model.

Please point out any other mistakes as well. Thank you.

$\endgroup$
1
+50
$\begingroup$

1) What notations to use for probability density function is it the one below: $\mathsf{P}_y(y_n|{\mathbf{u}_n})$ $\mathsf{P}_z(z_n|{\mathbf{u}_n})$ what goes in the subscript if I want to use $z$?

One way to write the density function is to subscripted upper case letters for the random variables. So $p_{Y_n}$ denotes the density of random variable $Y_n$ and $p_{Z_n}$ is the density of the random variable $Z_n$. Similarly, $p_\mathbf{Y}:=p_{Y_1,\ldots,Y_n}$ denotes the joint density of all the $Y_i$'s.

2) Is $P(\mathbf{y}|\mathbf{\theta})$ correct or $P_y(\mathbf{y}|\mathbf{\theta})$ or $P_z(\mathbf{z}|\mathbf{\theta})$?

See my answer to (1) above. In some cases it is ok to drop the subscript if it is understood which random variable the density $p$ belongs to. However, if you have lots of different random variables it is good to explicitly include the subscripts.

If I want to use expectation maximization, then would $\xi$ which denotes the complete data set consisting of the input signal and the received noisy signal be written as: $\xi = \{u_0,\ldots,u_{N-1},y_0,\ldots,y_{N-1} \}$ or $\xi = \{\hat u_0,\ldots,\hat u_{N-1},y_0,\ldots,y_{N-1} \}$

It is not clear what the hats mean.

4) Would the posterior be written as

You need to clarify the posterior of what conditioned on what. I assume you are interested in estimating the $u$'s so the posterior you are referring to is

\begin{eqnarray} p_{\mathbf{U}|\mathbf{Y}}(\mathbf u|\mathbf y) &\propto& p_{\mathbf{Y}|\mathbf{U}}(\mathbf y|\mathbf u) p_\mathbf{U}(\mathbf u) \\ &=& p_{Y_1,\ldots,Y_N|U_1,\ldots,U_{N}}(y_1,\ldots, y_N|u_1,\ldots u_N) \; p_{U_1,\ldots,U_{N}}(u_1,\ldots u_N)\\ &=& \left(\prod_{i=1}^N p_{Y_i | U_{i},\ldots U_{1 \wedge (i-p+1)}} (y_i|u_{i},\ldots u_{1\wedge (i-p+1)}) \right) \; p_{U_1,\ldots,U_{N}}(u_1,\ldots u_N). \end{eqnarray}

That $\wedge$ symbol denotes a min. operation and makes sure that the subscripts don't go below 1. If you have a probabilistic model for the $U_i$'s you can further factor out $p_{U_1,\ldots,U_{N}}$ into a product of conditional densities. For example, if it's a 1st order Markov model with a stationary initial distribution $p_{U_1}(\cdot)$ you can write, $$ p_{U_1,\ldots,U_{N}}(u_1,\ldots,u_N) = p_{U_1}(u_1)\;\prod_{i=2}^N p_{U_i|U_{i-1}}(u_i|u_{i-1}). $$

pdf, likelihood, log likelihood

Finally, I think, you may be interested in writing the "complete data" log-likelihood function i.e. the density of $\xi$ parameterized by $\theta$. This can be written as follows:

\begin{eqnarray} p_{\xi|\theta}(\xi|\theta) &=& p_{U_1,\ldots ,U_N,Y_1,\ldots ,Y_N|\theta}(u_1,\ldots u_N, y_1,\ldots y_N | \theta) \\ &=& p_{Y_1,\ldots ,Y_N | U_1,\ldots ,U_N,\theta}(y_1,\ldots y_N | u_1,\ldots u_N, \theta) \; p_{U_1,\ldots ,U_N|\theta}(u_1,\ldots u_N| \theta) \end{eqnarray}

and simplify this further using your observation model for $Y_i$'s and Markov transition model for $U_i$'s.

It seems that your confusion stems from being unfamiliar with how random variable notation works. I would recommend reviewing a textbook on probability and random processes for electrical engineers.

$\endgroup$
  • $\begingroup$ Why do you think $z$ is going to help with "simplification"? In the end you must deal with $\mathbf y$ and $\mathbf u$ directly. But if you insist, you can write $p_{Y_i|U_i,\ldots U_{i-p+1}}(y_i|u_i,\ldots u_{i-p+1}) = \mathcal{N}(z_i, \sigma_w^2)$ where $z_i = \mathbf h^T \mathbf u_i$. $\endgroup$ – Atul Ingle Oct 30 '17 at 19:50
  • $\begingroup$ I think you are planning to estimate model parameters ($\mathbf h$ and $\sigma_w^2$) using EM algorithm. You can denote these estimates with hats if you like. $\endgroup$ – Atul Ingle Oct 30 '17 at 20:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.