3
$\begingroup$

Say you want to compute the least squares estimate of $w$ from a data-set: $$ \begin{bmatrix}d_1 \\d_2 \\\vdots\\d_N \end{bmatrix} =\begin{bmatrix} x_1 \\x_2 \\ \vdots \\x_N\end{bmatrix}w + \begin{bmatrix} q_1 \\q_2 \\ \vdots \\q_N\end{bmatrix} $$ Note: $q$ is white noise. This can be compactly written as $$ \mathbf{d}= \mathbf{H}w + \mathbf{q} $$ The least squares estimate of $w$ which I shall call $w^*$ is given by \begin{align} w^* =& (\mathbf{H}^T\mathbf{H})^{-1}\mathbf{H}\mathbf{d}\\ = & \Big(\sum_{i =1} ^{N}x_i^2 \Big) ^{-1} \Big(\sum_{i =1} ^{N}x_id_i \Big) \end{align}

The Problem:

I have $N/2$ of the sample and my friend has the other half of the data set. We compute the least squares estimate separately. How do we now combine our estimates of $w^*$ besides 'averaging' it?

| improve this question | |
$\endgroup$
5
$\begingroup$

I'm unsure if this is the answer you are looking for, but why not save and share $|H_i|^2$ in addition to the least squares estimates?

If you have $w_1^*=\frac 1 {|H_1|^2} H_1^t d_1$, $w_2^*=\frac 1 {|H_2|^2} H_2^t d_2$, and also know $|H_1|^2$ and $|H_2|^2$, you get the total least squares estimate with:

$w^*=\frac 1 {|H_1|^2+|H_2|^2} (H_1^t d_1 + H_2^t d_2)=\frac{|H_1|^2}{|H_1|^2+|H_2|^2}w_1^*+\frac{|H_2|^2}{|H_1|^2+|H_2|^2}w_2^*$.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks! This is interesting. But what if I only want to share $w_1$ and $w_2$ $\endgroup$ – ssk08 Jul 9 '13 at 14:08
  • $\begingroup$ If you know that all $x_i$ are drawn from the same population, you could do something like taking your own (calculated) $|H_1|^2$ and for $|H_2|^2$ the expected value. If the distribution is unknown, you can bootstrap the expected value of $|H_2|^2$. Maybe someone else can give you a more accurate answer, I'm not an expert. $\endgroup$ – Wolf Jul 9 '13 at 14:54
0
$\begingroup$

A variant of the averaging would be to add the data sets before solving for w. This would automatically weight the result by favoring the data with more signal power.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.