# Questions on Weighted Least Square Estimation

This is a page from the book linear algebra,geodesy and gps by Gilbert Strang.... the page explains about the justification of the inverse of the of the co variance matrix of measurement vector $b$ in the over determined system $Ax = b$ as the best weight matrix for best estimate of x.

1. what does the line errors contained in the matrix mean?(underlined in blue color..)
2. how does that substitution take place marked with a blue curve...
3. is E{$rr^T$} = E{$bb^T$} ??

any explanation is welcome...

• Do not cross-post the same question to multiple sites (or leave links). So far you have posted this here, on Cross Validated and Mathematica... If you keep doing this, you might end up getting suspended. Commented Sep 29, 2012 at 16:54

You observe the vector $$\mathbf{b}$$ which is given by a linear transformation of the unknown data $$\mathbf{x}$$ plus noise (measurement error) $$\mathbf{r}$$:

$$\mathbf{b}=\mathbf{A}\mathbf{x}+\mathbf{r}\tag{1}$$

The error $$\mathbf{r}$$ is modeled as a random vector with known mean and known covariance. Note that the transformation matrix $$\mathbf{A}$$ is deterministic and known, and the data $$\mathbf{x}$$ are deterministic but unknown. Consequently, we have

$$E\left\{\mathbf{b}\right\}=E\left\{\mathbf{Ax}+\mathbf{r}\right\}=\mathbf{Ax}+E\{\mathbf{r}\}=\mathbf{Ax}\tag{2}$$

since the mean of the error is known to be $$E\{\mathbf{r}\}=0$$.

Eqs $$(1)$$ and $$(2)$$ imply

$$\mathbf{b}-E\{\mathbf{b}\}=\mathbf{r}\tag{3}$$

What is meant by the underlined sentence in your question is that all we need to know about the error $$\mathbf{r}$$, apart from the given mean, can be deduced from the observation $$\mathbf{b}$$. And the only thing we need to know in addition to the mean is the covariance matrix.

With the given assumptions, the covariance of the observation equals the covariance of the error:

$$\mathbf{\Sigma}_b=E\left\{(\mathbf{b}-E\{\mathbf{b}\})(\mathbf{b}-E\{\mathbf{b}\})^T\right\}=E\{\mathbf{r}\mathbf{r}^T\}=\mathbf{\Sigma}_r\tag{4}$$

where I've used $$(3)$$.

1. The linear model is $b=Ax+r$, so you can see the errors $r$ as the noise added to the measurements $b$.

2. The covariance $\Sigma_b=E\{(b-\bar{b})(b-\bar{b})^T\}=E\{(b-Ax)(b-Ax)^T\}=E\{rr^T\}$

3. $E\{rr^T\} \neq E\{bb^T\}$ unless $Ax=0$

• 1...can you explain what is this $\hat{b}$ ? is it the estimated value like $\hat{x}$ if yes then where do you get that from ? or is it the mean of $b$ ? 2...how is $\hat{b} = Ax$?? Commented Sep 29, 2012 at 13:00
• Sorry, what I mean is $\bar{b}$, which is the mean of $b$. Since $r$ is assumed to be zero mean, then $\bar{b}=Ax$. Commented Sep 29, 2012 at 13:48
• $E(b) = E(Ax + r)$....$E(b) = E(Ax) + E(r)$....$\bar{b} = AE(x)+ E(r)$....$\bar{b} = A\bar{x} + 0$....is $A\bar{x} = Ax$ ?? Commented Sep 29, 2012 at 13:51
• For BLUE, $x$ is deterministic Commented Sep 29, 2012 at 14:15
• i think for BLUE $E(x - \hat{x}) = 0$ not $\bar{x} = x$ Commented Sep 29, 2012 at 14:46