Skip to main content
edited tags
Link
Royi
  • 20.5k
  • 4
  • 199
  • 240
added 576 characters in body; edited tags
Source Link
Royi
  • 20.5k
  • 4
  • 199
  • 240

Given $ M \in {R}^{NxN} $$ M \in \mathbb{R}^{N \times N} $ which is a Positive Definite Matrix.
Let $ \hat{x} $ the MMSE of $ x $ given $ z $, namely $ \hat{x} = \mathbb{E} \left[ x \mid z \right] $.

How comeProve the following are equivalentequivenalce of minimizing $ E[{(\hat{x} - x)}^{T}(\hat{x} - x)] $the following expressions:

  1. $ E[{(\hat{x} - x)}^{T} M (\hat{x} - x)] $

    $ \arg \min_{\hat{x}} \mathbb{E} \left[ {\left( \hat{x} - x \right)}^{T} \left( \hat{x} - x \right) \mid z \right] $.

  2. $ trace(M E[(\hat{x} - x) {(\hat{x} - x)}^{T}]) $

    $ \arg \min_{\hat{x}} \mathbb{E} \left[ {\left( \hat{x} - x \right)}^{T} M \left( \hat{x} - x \right) \mid z \right] $.

  3. $ \arg \min_{\hat{x}} \operatorname{Tr} \left( M \mathbb{E} \left[ {\left( \hat{x} - x \right)}^{T} \left( \hat{x} - x \right) \mid z \right] \right) $

I can seeEquivalence means all are actually minimized by $ \hat{x} = \mathbb{E} \left[ x \mid z \right] $.

The equivalence of 2 and 3 is easy using the intuition in keeping "Quadratic" form yet can't seeCyclic Property of the formal proof$ \operatorname{Tr} \left( \cdot \right) $ operator.
Yet showing the invertible transformation keeps the solution (Equivalence of 1 and 2) isn't trivial.

Given $ M \in {R}^{NxN} $ Positive Definite Matrix.
Let $ \hat{x} $ the MMSE of $ x $ given $ z $.

How come the following are equivalent of minimizing $ E[{(\hat{x} - x)}^{T}(\hat{x} - x)] $:

  1. $ E[{(\hat{x} - x)}^{T} M (\hat{x} - x)] $
  2. $ trace(M E[(\hat{x} - x) {(\hat{x} - x)}^{T}]) $

I can see the intuition in keeping "Quadratic" form yet can't see the formal proof.

Given $ M \in \mathbb{R}^{N \times N} $ which is a Positive Definite Matrix.
Let $ \hat{x} $ the MMSE of $ x $ given $ z $, namely $ \hat{x} = \mathbb{E} \left[ x \mid z \right] $.

Prove the equivenalce of the following expressions:

  1. $ \arg \min_{\hat{x}} \mathbb{E} \left[ {\left( \hat{x} - x \right)}^{T} \left( \hat{x} - x \right) \mid z \right] $.

  2. $ \arg \min_{\hat{x}} \mathbb{E} \left[ {\left( \hat{x} - x \right)}^{T} M \left( \hat{x} - x \right) \mid z \right] $.

  3. $ \arg \min_{\hat{x}} \operatorname{Tr} \left( M \mathbb{E} \left[ {\left( \hat{x} - x \right)}^{T} \left( \hat{x} - x \right) \mid z \right] \right) $

Equivalence means all are actually minimized by $ \hat{x} = \mathbb{E} \left[ x \mid z \right] $.

The equivalence of 2 and 3 is easy using the Cyclic Property of the $ \operatorname{Tr} \left( \cdot \right) $ operator.
Yet showing the invertible transformation keeps the solution (Equivalence of 1 and 2) isn't trivial.

Tweeted twitter.com/#!/StackSignals/status/451627260060368896
Source Link
Royi
  • 20.5k
  • 4
  • 199
  • 240

Minimum Mean Square Estimator - Equivalent Expressions to Minimize

Given $ M \in {R}^{NxN} $ Positive Definite Matrix.
Let $ \hat{x} $ the MMSE of $ x $ given $ z $.

How come the following are equivalent of minimizing $ E[{(\hat{x} - x)}^{T}(\hat{x} - x)] $:

  1. $ E[{(\hat{x} - x)}^{T} M (\hat{x} - x)] $
  2. $ trace(M E[(\hat{x} - x) {(\hat{x} - x)}^{T}]) $

I can see the intuition in keeping "Quadratic" form yet can't see the formal proof.