1
$\begingroup$

So I'm trying to derive an analytical solution for a MLE that should estimate a static value polluted by multiplicative Gaussian noise.

The vector of measurements $\tilde{\boldsymbol{d}}$ is given as $a[n]x_u$ where $a[n]$ is the $n$-th realization of the random variable $A \propto \mathcal{N}(\mu_A, \sigma_A)$ and $x_u$ is an unknown constant that should be estimated from $\tilde{\boldsymbol{d}}$.

I got as far as the log-likelihood function, which I now need to maximize with regard to $x$. To do this, I need to take the derivative of the log-likelihood , set it to zero and solve for $x$. However, the summation term at the end is giving me a headache as I can't figure out its derivative.

$\ell(x, \tilde{\boldsymbol{d}}) = -N \cdot \log x - N\cdot \log\sigma_A - \frac{N}{2} \log 2\pi - \frac{1}{2\sigma^2_A}\sum\limits_{n=1}^N\bigg(\frac{\tilde{d}[n]}{x}-\mu_A\bigg)^2$

| improve this question | |
$\endgroup$
  • $\begingroup$ Could you include the distribution of your "static value polluted by multiplicative Gaussian noise" ? $\endgroup$ – Gilles Jul 8 '16 at 19:16
  • 1
    $\begingroup$ @Gilles I have edited the question to include the original problem. $\endgroup$ – sobek Jul 9 '16 at 11:38
  • $\begingroup$ Thanks ! I think you meant $A \sim \mathcal N (\mu_A, \sigma^2_a)$. $\endgroup$ – Gilles Jul 9 '16 at 14:55
  • $\begingroup$ Btw is $\tilde d$ given as $a[n] x_u$ with $x_u$ fixed or is given as $a[n] x$ with $x$ variable ? $\endgroup$ – Gilles Jul 9 '16 at 15:21
  • $\begingroup$ please see my answer below. $\endgroup$ – Gilles Jul 9 '16 at 16:01
4
$\begingroup$

OK, let's have a look at one of the problematic terms: $$ \frac{\delta}{\delta x} \bigg[ \bigg(\frac{\tilde{d}[n]}{x}-\mu_A\bigg)^2 \bigg ] = - \frac{2 \tilde{d}[n] \bigg (\tilde{d}[n] - \mu_A x\bigg) }{x^3} $$ which can be verified by Wolfram Alpha.

The full derivative of the summation term is then just this summed over $n$.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Ah yes, of course all summation terms can be treated independently. Thanks! $\endgroup$ – sobek Jul 9 '16 at 11:50
1
$\begingroup$

To be clear:

I'm assuming $\mathbf{\tilde{d}}$ is given as $a[n]x$, with a deterministic variable $x$, so that with the i.i.d. on $\mathbf{\tilde{d}}$ you then have $$ \text{if}\quad A \sim \mathcal (\mu_A, \sigma^2_A) \Longrightarrow \mathbf{\tilde{d}} \sim \mathcal (\mu_A x, \sigma^2_A x^2) $$ And with this, only the first and last term of your log-likelihood depend on $x$; I'm assuming your $\log(\cdot)$ is the natural logarithm $\ln (\cdot)$, otherwise you'll have to adjust the quadratic term accordingly. And this then gives you \begin{align} \frac{\textrm{d}\left[\ell(x, \tilde{\boldsymbol{d}})\right]}{\textrm{d}x}&=\frac{-N}{x}-\frac{1}{2\sigma^2_A}\sum_{n=1}^N\left[ -2\frac{\tilde{d}[n]}{x^2}\left(\frac{\tilde{d}[n]}{x}-\mu_A\right)\right]\\ &=\frac{-N}{x}+\frac{1}{\sigma^2_A{x^3}}\sum_{n=1}^N\tilde{d}[n]\left(\tilde{d}[n]-\mu_A x\right) \end{align} You can proceed from here.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Why do you say that all x in the log-likelihood would become $x_u$? This does not make sense to me. $x_u$ is only contained in $\tilde{\boldsymbol{d}}$. Also $x$ is not a function of $n$. If our estimator is unbiased, then at the maximum of $\ell$ with regard to $x$, $x$ is the estimate of $x_u$. $\endgroup$ – sobek Jul 9 '16 at 17:20
  • $\begingroup$ @sobek you're right, $x$ is not a function of $n$. And wrong formulation. However, if $x_u$ is to be estimated then your PDF is a function $x$ (and not $x_u$), $\mu_A$, and $\sigma^2_A$. $\endgroup$ – Gilles Jul 9 '16 at 19:59
  • $\begingroup$ The PDF of the estimator, yes, but the PDF of $\tilde{\boldsymbol{d}}$, this i don't really understand. Thanks for the hint about the logarithm, you are right, since it is only used to make it easier to deal with the exponential of the Gaussian PDF, the natural logarithm must be used. $\endgroup$ – sobek Jul 9 '16 at 20:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.