# How to Combine / Fuse 2 Least Squares Estimates?

Say you want to compute the least squares estimate of $w$ from a data-set: $$\begin{bmatrix}d_1 \\d_2 \\\vdots\\d_N \end{bmatrix} =\begin{bmatrix} x_1 \\x_2 \\ \vdots \\x_N\end{bmatrix}w + \begin{bmatrix} q_1 \\q_2 \\ \vdots \\q_N\end{bmatrix}$$ Note: $q$ is white noise. This can be compactly written as $$\mathbf{d}= \mathbf{H}w + \mathbf{q}$$ The least squares estimate of $w$ which I shall call $w^*$ is given by \begin{align} w^* =& (\mathbf{H}^T\mathbf{H})^{-1}\mathbf{H}\mathbf{d}\\ = & \Big(\sum_{i =1} ^{N}x_i^2 \Big) ^{-1} \Big(\sum_{i =1} ^{N}x_id_i \Big) \end{align}

The Problem:

I have $N/2$ of the sample and my friend has the other half of the data set. We compute the least squares estimate separately. How do we now combine our estimates of $w^*$ besides 'averaging' it?

• Also asked on math.SE. Jul 9 '13 at 13:51

I'm unsure if this is the answer you are looking for, but why not save and share $|H_i|^2$ in addition to the least squares estimates?
If you have $w_1^*=\frac 1 {|H_1|^2} H_1^t d_1$, $w_2^*=\frac 1 {|H_2|^2} H_2^t d_2$, and also know $|H_1|^2$ and $|H_2|^2$, you get the total least squares estimate with:
$w^*=\frac 1 {|H_1|^2+|H_2|^2} (H_1^t d_1 + H_2^t d_2)=\frac{|H_1|^2}{|H_1|^2+|H_2|^2}w_1^*+\frac{|H_2|^2}{|H_1|^2+|H_2|^2}w_2^*$.
• Thanks! This is interesting. But what if I only want to share $w_1$ and $w_2$ Jul 9 '13 at 14:08
• If you know that all $x_i$ are drawn from the same population, you could do something like taking your own (calculated) $|H_1|^2$ and for $|H_2|^2$ the expected value. If the distribution is unknown, you can bootstrap the expected value of $|H_2|^2$. Maybe someone else can give you a more accurate answer, I'm not an expert.