I am learning estimation theory through Steven M. Kay - Fundamentals of Statistical Signal Processing, Volume 1: Estimation Theory. In the Chapter 12 (Linear Bayesian Estimator), Theorem 12.1 (Bayesian Gauss-Markov Theorem) gives the LMMSE estimation of the signal based on the linear noisy measurement under the Gaussian prior assumption:
If the data are described by the Bayesian linear model form \begin{equation} \boldsymbol{x}=\boldsymbol{H\theta}+\boldsymbol{w} \tag{12.25} \end{equation} where $\boldsymbol{x}$ is an $N \times 1$ data vector, $\boldsymbol{H}$ is a known $N\times p$ observation matrix, $\boldsymbol{\theta}$ is a $p \times 1$ random vector of parameters whose realization is to be estimated and has mean $E(\boldsymbol{\theta})$ and covariance matrix $\boldsymbol{C}_{\theta\theta}$, and $\boldsymbol{w}$ is an $N \times 1$ random vector with zero mean and covariance matrix $\boldsymbol{C}_w$ and is uncorrelated with $\boldsymbol{\theta}$ (the joint PDF $p(\boldsymbol{w},\boldsymbol{\theta})$ is otherwise arbitrary), then the LMMSE estimator of $\boldsymbol{\theta}$ is \begin{align} \hat{\boldsymbol{\theta}} & = E(\boldsymbol{\theta})+\boldsymbol{C}_{\theta\theta}\boldsymbol{H}^T(\boldsymbol{H}\boldsymbol{C}_{\theta\theta}\boldsymbol{H}^T+\boldsymbol{C}_w)^{-1}(\boldsymbol{x}-\boldsymbol{H}E(\boldsymbol{\theta})) \tag{12.26} \\ & = E(\boldsymbol{\theta})+(\boldsymbol{C}_{\theta\theta}^{-1}+\boldsymbol{H}^T\boldsymbol{C}_w^{-1}\boldsymbol{H})^{-1}\boldsymbol{H}^T\boldsymbol{C}_w^{-1}(\boldsymbol{x}-\boldsymbol{H}E(\boldsymbol{\theta})) \tag{12.27} \end{align} The performance of the estimatior is measured by the error $\boldsymbol{\epsilon}=\boldsymbol{\theta}-\hat{\boldsymbol{\theta}}$ whose mean is zero and whose covariance matrix is \begin{align} \boldsymbol{C}_\boldsymbol{\epsilon} &= E_{\boldsymbol{x},\boldsymbol{\theta}}(\boldsymbol{\epsilon}\boldsymbol{\epsilon}^T) \\ & = \boldsymbol{C}_{\theta\theta} - \boldsymbol{C}_{\theta\theta}\boldsymbol{H}^T(\boldsymbol{H}\boldsymbol{C}_{\theta\theta}\boldsymbol{H}^T+\boldsymbol{C}_w)^{-1}\boldsymbol{H}\boldsymbol{C}_{\theta\theta} \tag{12.28} \\ & = (\boldsymbol{C}_{\theta\theta}^{-1}+\boldsymbol{H}^T\boldsymbol{C}_w^{-1}\boldsymbol{H})^{-1} \tag{12.29} \end{align}
Since the prior of $\boldsymbol{\theta}$ is Gaussian, the LMMSE estimate $\hat{\boldsymbol{\theta}}_{LMMSE}$ is equivalent to the MMSE estimate $\hat{\boldsymbol{\theta}}_{MMSE}$, and $\hat{\boldsymbol{\theta}}_{MMSE}$ is equal to the posterior mearn $E(\boldsymbol{\theta}|\boldsymbol{x})$. Since the prior and likelihood are both Gaussian, the posterior distribution $p(\boldsymbol{\theta}|\boldsymbol{x})$ is also Gaussian.
Here I am trying to derive $\hat{\boldsymbol{\theta}}_{MMSE}$ and $\boldsymbol{C}_\boldsymbol{\epsilon}$ from the perspective of PDF multiplication, that is, calculate $p(\boldsymbol{\theta}|\boldsymbol{x}) \propto p(\boldsymbol{x}|\boldsymbol{\theta})p(\boldsymbol{\theta})=\mathcal{N}(\boldsymbol{x};\boldsymbol{H\theta},\boldsymbol{C}_{w})\mathcal{N}(\boldsymbol{\theta};E(\boldsymbol{\theta}),\boldsymbol{C}_{\theta\theta})$, and formulate the quadratic and firse-order terms of $\boldsymbol{\theta}$ at the exponential to form a Gaussian PDF. The covariance matrix of $p(\boldsymbol{\theta}|\boldsymbol{x})$ I got matches 12.29, but the posterior mean is the following form:
\begin{equation}
E(\boldsymbol{\theta}|\boldsymbol{x}) = \boldsymbol{C}_{\boldsymbol{\epsilon}}(\boldsymbol{H}^T\boldsymbol{C}_w^{-1} \boldsymbol{x}+\boldsymbol{C}_{\theta\theta}^{-1}E(\boldsymbol{\theta})) \tag{q1}
\end{equation}
So my question is, is the posterior mean I got in q1 equal to the $\hat{\boldsymbol{\theta}}$ given in 12.26 and 12.27? If so, how can I reach that?
By the way, I can't find the way from 12.26 to 12.27 (12.28 to 12.29 either). So can someone give me a hint?
