2
$\begingroup$

Taken from Guide to DSP where it says:

... at two ... standard deviations from the mean, the value of the Gaussian curve has dropped to about 1/19 ...

It seems to be a straight forward calculation but my math just didn't work out to 1/19. The Gaussian probability distribution function is given by:

$$ P(x) = {1 \over \sqrt{2\pi} \sigma } e^{-{{(x - \mu)^2} \over {2 \sigma^2}}} $$

At the mean (where maximum probability occurs),

$$ \begin{align} P(\mu) &= {1 \over \sqrt{2\pi} \sigma } e^{-{{(\mu - \mu)^2} \over {2 \sigma^2}}} \\ &= {1 \over \sqrt{2\pi} \sigma } e^{0} = {1 \over \sqrt{2\pi} \sigma} \end{align} $$

And at 2 standard deviation, or $ x = \mu +2\sigma $

$$ \begin{align} P(\mu + 2\sigma) &= {1 \over \sqrt{2\pi} \sigma } e^{-{{(\mu + 2\sigma - \mu)^2} \over {2 \sigma^2}}} \\ &= {1 \over \sqrt{2\pi} \sigma } e^{-{{4\sigma^2} \over {2 \sigma^2}}} = {1 \over \sqrt{2\pi} \sigma } e^{-2} \end{align} $$

To see how much probability has dropped from max probability at $ \mu $ to probability at $ 2\sigma $, I can simply take the ratio:

$$ { P(\mu + 2 \sigma) \over P(\mu) } = { {{1 \over {\sqrt{2\pi}\sigma}} e^{-2}} \over {{1 \over {\sqrt{2\pi}\sigma}}}} = e^{-2} = 0.135 \approx {2 \over 15} \neq {1 \over 19} $$

$\endgroup$
2
  • $\begingroup$ Yeah, that seems like a mistake. Since he publishes errata as well, maybe drop the author an email. $\endgroup$
    – mmmm
    Aug 1 at 7:10
  • 4
    $\begingroup$ The blog must have referred to the standard normal distribution, i.e. the Gaussian curve with $\mu=0$ and $\sigma=1$. In that case, the values of the PDF at 2, 4, and 6 are 1/19, 1/7563, and 1/166,666,666 respectively. I agree that the wording of the blog is confusing. $\endgroup$
    – AlexTP
    Aug 1 at 8:35
5
$\begingroup$

You've computed the most general case and have shown it's always 2/15, thus 1/19 is incorrect, at least if interpreting "1/x-th of value" as $f(\text{value})/f_\text{max}$. This simulation confirms it.

Edit: I took a guess, 1/19 is the value of the Gaussian at two standard deviations for $(\mu, \sigma) = (0, 1)$, which does qualify per exact wording of "the value drops to 1/19". But I agree your measure is more meaningful as it's independent of $(\mu, \sigma)$ and interprets as "the value drops to 2/15 of its peak value".

$\endgroup$
3
$\begingroup$

As noted by AlexTP and OverLordGoldDragon, the confusion is that the actual value of the Gaussian probability density curve is 1/19, while the value compared to the 0.4 peak value is 2/15.

When talking about probability distributions, the middle or peak value does not have a particular significance. The absolute value of PDF is more useful in practice than the ratio to peak value.

For example, a distribution that has a wide flat peak with sharp edges would have a lower peak probability, for example 0.2. If it has density of 1/19 at 2 sigma, it has the same probability of having outliers that far as the Gaussian distribution would. But if you used the comparison to peak value, you would think it was less sharp drop than the Gaussian, just because the middle is more flat.


As an example, let's compare the Laplace distribution to Gaussian (normal) distribution:

Distribution comparison (Image source)

The ratio of value at 2 standard deviations compared to peak value is 0.1 for both distributions. The actual probability of events outside 2 sigma is lower for Gaussian distribution, while Laplace distribution does have steeper descent at the middle.

The discussion in the original article was that in Gaussian distribution the "tails drop toward zero very rapidly", where tails means the long portion further away from middle.

$\endgroup$
17
  • 1
    $\begingroup$ 1/19 by itself provides no information on extent of decay, whereas 2/15 does. Only if another absolute value is provided can one infer about rate of decay. $\endgroup$ Aug 1 at 17:13
  • 2
    $\begingroup$ @OverLordGoldDragon Also, what about a uniform distribution whose support is $[-M\sigma,+M\sigma]$ where $\sigma^2$ is the variance of the normal distribution of interest. The ratio "mean to (every point of the support)" is $1$, and I can choose $M$ to make the PDF value as small as I want. The ratio is not useful without assuming a class of distribution. $\endgroup$
    – AlexTP
    Aug 3 at 8:39
  • 1
    $\begingroup$ @OverLordGoldDragon (1) No, the OP's linked context wants to justify "In practice, the sharp drop of the Gaussian pdf dictates that these extremes almost never occur." by the rate of decay, which is confusing and simply misleading. In the context of the OP's question and the referenced blog, the ratio is only useful to compare two normal distributions. (2) I don't say that the ratio is always useless because if you want to use it, it is useful. But, again, not in the context of this specific question of the OP. $\endgroup$
    – AlexTP
    Aug 3 at 12:14
  • 1
    $\begingroup$ @AlexTP Absolute value at $s$ standard deviations changes, but not ratio to peak; since this holds at all points and for every $(\mu, \sigma)$, it illustrates invariance. $\endgroup$ Aug 4 at 9:23
  • 2
    $\begingroup$ @OverLordGoldDragon and (2) Isn't the invariance just a consequence of "the affine transformation of a normally distributed random variable is a normally distributed random variable"? For (2), to avoid writing down rigorous proof, just think that distribution is fully defined by its PDF. I mean your statement about the invariance may be a nice property, but not many find it useful, sorry. You may want to create a new question or answer to avoid extending this comment section. $\endgroup$
    – AlexTP
    Aug 4 at 10:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.