3
$\begingroup$

I'm reading the paper at the link below and I was following it for about 2 pages until I hit a road block on the bottom of page 648 where the author says:

putting together 9-11, we obtain

and gives equation 12. I understand equations 9, 10 and 11 but don't see how 12 comes about from them.

Also, if anyone knows a book or a paper that provides an exact mathematical definition of white noise, it's appreciated. Definitely, I'm seeing how important it is to understand white noise, particularly in the context of this paper. Gaussanity ( which is what I'm used to seeing in the many derivations of the Kalman filter ) is clearly quite different ( Dilip, I always believed you but now I'm seeing the importance ) and doesn't cut it here. This paper is actually the first one of a four part-er but, if I can follow this one by the end of 2018, I'd be quite satisfied.

An Innovations Approach to Least Squares Estimation Part I: Linear Filtering in Additive White Noise.

Improve this question
$\endgroup$
4
  • $\begingroup$ Hi Mark, as I had to read your question twice to tell content from politeness, I just removed a bit of the less information-carrying part of your question, hoping to enhance its readability. $\endgroup$
    – Marcus Müller
    Jul 6, 2018 at 11:15
  • $\begingroup$ The definition of "white noise" is a stochastic signal $X$ with an autocorrelation function (ACF) $R_X(t_1,t_2)$ that is $0$ everywhere but for $t_1=t_2$, for all matters relevant here. The condition that this signal still should have non-zero energy implies the ACF takes shape of a scaled dirac delta distribution$\delta(t_1-t_2)$. That directly leads to the Power Spectral Density (PSD) being a constant (which is the physical origin of it being called white, as "white light" contains all frequencies) $\endgroup$
    – Marcus Müller
    Jul 6, 2018 at 11:20
  • $\begingroup$ (I'd like to add that the autocorrelation function criterion described above is actually kind of only sufficient, depending on who you ask: I'd, myself, would argue that whiteness requires the stochastic process to be independent for different points of observation $t_1\ne t_2$, but the ACF only says they are uncorrelated – but meh, that really makes not much of a difference here, so others define it like above, and it's maybe less bulky than saying $f_X(t_1)\cdot f_X(t_2)=f_X(t_1,t_2) \forall t_1\ne t_2$ and definitily gives us something at least estimateable to decide whiteness.) $\endgroup$
    – Marcus Müller
    Jul 6, 2018 at 11:29
  • $\begingroup$ @Marcus Muller: Thanks Marcus. No problem with editing my question. Will read your comments carefully and slowly. $\endgroup$
    – mark leeds
    Jul 6, 2018 at 13:03

2 Answers 2

Reset to default
0
$\begingroup$

They used to have different convention for writing this stuff back then. But actually what you saw is really simple.

It's all based on the Orthogonal Principle of MMSE.

They say in (9) that the Additive Noise is uncorrelated (By defining its Auto Correlation by Delat Function).
In (10) they say the optimal estimation of $ x \left( t \right) $ is given by $ \hat{x} \left( t \mid t \right) $ is a linear combination of $ y \left( t \right) $ and $ z \left( t \right) $.
In (11) they exactly use the Orthogonal Principle which the estimator must obey.

In (12) they just derive the Correlation Matrix between the processes.
Since the linear estimator represent the correlation it is not surprising to see that it is the linear combination defined by the optimal filter $ g \left( \cdot \right) $.

The trick here is the fact (Using article notations) $ \overline{ \left( x \left( t \right) - \hat{x} \left( t \mid t \right) \right) v' \left( s \right) } = 0 $ which means (By Linearity of the Expectation) $ \overline{ x \left( t \right) v' \left( s \right) } = \overline{ \hat{x} \left( t \mid t \right) v' \left( s \right) } $.
Expand the right hand term by (10) by plugging $ \hat{x} \left( t \mid t \right) $ and you get (12).

By the way, Kaliath has much better written, in my opinion, paper on the subject called A View of Three Decades of Linear Filtering Theory.

Improve this answer
$\endgroup$
1
  • $\begingroup$ That was tremendous. My problem was the "trick here is the fact" part. Thank you so much, especially for going through it and bothering to understand all the notation etc. I know about and have Kalaith's three decades treatise but these looked less intimidating. I'll check it out more closely given what you said. $\endgroup$
    – mark leeds
    Jul 7, 2018 at 15:12
1
$\begingroup$

If I might offer a partial answer to

Also, if anyone knows a book or a paper that provides an exact mathematical definition of white noise, it's appreciated.

The book that was popular at he time that Kalaith was a grad student at MIT was

Doob, Joseph L. Stochastic processes. Vol. 7. No. 2. New York: Wiley, 1953.

This book is often cited by other MIT contemporaries of that period, so I would guess that this is the kernal of what Kalaith conceived as white noise.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Hi Stanley: That's a very popular book in the stat world also, I don't have it but do plan on purchasing it in this century, I found a very detailed answer by Dilip that I will post in a moment. Thanks for mentioning Doob. I kind of forgot about him. $\endgroup$
    – mark leeds
    Jul 6, 2018 at 16:13
  • $\begingroup$ My mistake. Detailed answer is here and provided by Ben CW. It will take me a while to digest but it looks interesting. math.stackexchange.com/questions/134193/… $\endgroup$
    – mark leeds
    Jul 6, 2018 at 16:20
  • $\begingroup$ When I say answer above, I'm only referring to definition of white noise. $\endgroup$
    – mark leeds
    Jul 6, 2018 at 16:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.