1
$\begingroup$

How can we prove the auto-correlation function of white gaussian noise $\{ R_{N(t)}(\tau) \}$ is $\frac{N_0}{2} \delta(\tau)$ from its p.d.f in equation below? $$ f_{N(t)}(\eta)=\frac{1}{\sqrt{2 \pi \sigma^2}}\exp\left(\frac{-(\eta-\mu)^2}{2 \sigma^2}\right) $$

My Approach:

We know: $\frac{N_0}{2}=\sigma^2 \quad(\text{ or variance}) \tag{1}$ We also know: $$ R_{X(t)}(\tau)= E[X(t)X(t+ \tau)]=\int_{y=- \infty}^{\infty} \int_{x= - \infty}^{\infty} x \space y \space f_{X(t) X(t+\tau)}(x,y) \space dx \space dy $$ Where $f_{X(t) X(t+\tau)}(x,y) \text{ is joint p.d.f}$ $$\text{so , } R_{N(t)}(\tau)= E[N(t)N(t+ \tau)]=\int_{y=- \infty}^{\infty} \int_{x= - \infty}^{\infty} x \space y \space f_{N(t)N(t+\tau)}(x,y) \space dx \space dy$$ Now we know $$f_{N(t)}(\eta)=\frac{1}{\sqrt{2 \pi \sigma^2}}\exp\left(\frac{-(\eta-\mu)^2}{2 \sigma^2}\right)$$
then, how can we find $f_{N(t)N(t+\tau)}(x,y)$ ?

$\endgroup$
  • $\begingroup$ Hint : You can use the definition of a white noise process. That is, all frequency components are present in the power spectral density (PSD) $\endgroup$ – AnVij Mar 16 '18 at 6:15
  • $\begingroup$ yeah I know that process @AnVij ; but i want to derive its auto-correlation function from its p.d.f $\endgroup$ – Suresh Mar 16 '18 at 7:35
  • 2
    $\begingroup$ A Gaussian PDF does not imply whiteness, so you can't derive the auto-correlation function just from the PDF. $\endgroup$ – Matt L. Mar 16 '18 at 9:09
3
$\begingroup$

As @MattL points out in a comment, a Gaussian pdf does not imply whiteness. Indeed, it can be argued that the assumption that the process is a continuous-time white noise process is contrary to the belief that the random variables constituting the process even have a pdf of any kind. Continuous-time white noise is a mythical beast that is used to account for the empirical observation that the power spectral density $S_{\scriptstyle{\text{output}}}(f)$ of the noise at the output of a linear filter with transfer function $H(f)$ can be expressed as $$S_{\scriptstyle{\text{output}}}(f) = \frac{N_0}{2}|H(f)|^2,-\infty < f < \infty,$$ a result that is perfectly consistent with the standard theory of wide-sense-stationary processes in linear systems which claims that $$S_{\scriptstyle{\text{output}}(f)} = S_{\scriptstyle{\text{input}}(f)}|H(f)|^2, -\infty < f < \infty,$$ provided that we are willing to suspend belief and assume that the input noise has power spectral density $\frac{N_0}{2}, -\infty < f < \infty,$ which has infinite power. (This suspension of belief must be accompanied by a most solemn vow to not attempt to solve the world's energy crisis by plugging in the power grid into a white noise generator!). Formally, a continuous-time white noise process is defined to be a continuous-time zero-mean wide-sense stationary process whose power spectral density has constant value for all choices of frequency $f$. Continuous-time white noise can only be observed through some kind of device (whether linear or nonlinear) which perforce performs some kind of filtering or bandwidth reduction so that the observation has finite variance and the observed process has finite power. The observed process is not called a white noise process.

The inverse Fourier transform of $\frac{N_0}{2}, -\infty < f < \infty,$ is the autocorrelation function $\frac{N_0}{2}\delta(t)$ of the white noise process where $\delta(t)$ is the Dirac delta. It is a shibboleth often murmured on this site that the value $\delta(0)$ of the Dirac delta is $\infty$ with the usual consequences for this mispronunciation being administered by the cognoscenti of dsp.SE, but the point is that we cannot define any kind of pdf for the random variables constituting a white noise process, and very certainly not the Gaussian pdf that the OP wishes to use. What is true that the noise at the output of the linear filter is Gaussian noise meaning that the random variables constituting the output noise process are (zero-mean) Gaussian random variables with variance $$\sigma^2 = \int_{-\infty}^\infty S_{\scriptstyle{\text{output}}(f)}\, \mathrm df = \frac{N_0}{2}\int_{-\infty}^\infty |H(f)|^2\mathrm df.$$ This result is also consistent with the view that the input noise process is white Gaussian noise in the sense that it is a standard result in the theory of wide-sense-stationary process that linear filtering of a Gaussian process results in a Gaussian process. But it is not possible to ascribe a variance (not even an infinite variance) to these alleged Gaussian random variables in the input process because for all $x$, $$\lim_{\sigma\to\infty} \frac{\exp\left(-\frac{x^2}{2\sigma^2}\right)}{{\sqrt{2\pi}\sigma}} = 0.$$

In summary, continuous-time white Gaussian noise is a myth used to provide a rationale for the empirical observation that the noise at the output of a linear filter with transfer function $H(f)$ is a zero-mean wide-sense-stationary (and hence strictly stationary) Gaussian random process with power spectral density $$S_{\scriptstyle{\text{output}}}(f) = \frac{N_0}{2}|H(f)|^2,-\infty < f < \infty.$$ The only assumption about the random variables constituting the wide-sense-stationary input white noise process is that they all are zero-mean random Gaussian random variables but we cannot ascribe any variance to them or observe them directly or do anything else with them until we have processed them through some device of finite bandwidth.

More than what you probably want to know about continuous-time white Gaussian noise can be found in the Appendix of this lecture note of mine.


In view of a veritable flood of comments from @markleeds on the original version of this answer, it is worth noting that on dsp.SE (and any place else where MATLAB is the lingua franca), there is also the concept of a discrete-time white noise process. A discrete-time process is a countable collection of random variables $\{X[k]\colon k \in \mathbb Z\}$ (notice the change from $X(t)$ to $X[k]$: this is very commonly-used notational convention to distinguish the two cases) and a discrete-time white noise process is just a collection of zero-mean finite-variance independent identically distributed random variables $N[k]$, $k \in \mathbb Z$. The (discrete-time) autocorrelation function of the process is $$R_N[k] = E[N[m]N[m+k]] = \sigma^2\delta[k]$$ where $\delta[k]$ is the Kronecker delta (not the Dirac delta) which is defined as $$\delta[k] = \begin{cases}1, & k=0,\\0, & k\neq 0.\end{cases}$$ Although the same symbol is used in both cases, we use $()$ and $[]$ as clues to tell us what $\delta$ means. Now, what, if any, is the relationship between samples of continuous-time white noise and discrete-time white noise? Well, as mentioned above, sampling white noise is not a physical reality -- one cannot draw blood from a beast that does not exist! -- and any attempt to do so must take into account the physical properties of the sampler which is a finite-bandwidth device with a switch that closes for a very short period of time $\varepsilon$ and then opens again. A simple model for what happens during this time is that what the sampler reads at time $t$ is the value of $$\int_{t-\varepsilon}^{t}N(\tau)\, \mathrm d\tau$$ which is a random variable with mean $0$ and variance \begin{align} \sigma^2 &= E\left[\int_{t-\varepsilon}^{t}N(\tau)\, \mathrm d\tau\int_{t-\varepsilon}^{t}N(\lambda)\, \mathrm d\lambda\right]\\ &= E\left[\int_{t-\varepsilon}^{t}\int_{t-\varepsilon}^{t}N(\tau)N(\lambda)\, \mathrm d\tau \,\mathrm d\lambda\right]\\ &= \int_{t-\varepsilon}^{t}\int_{t-\varepsilon}^{t}E[N(\tau)N(\lambda)]\, \mathrm d\tau \,\mathrm d\lambda\\ &= \int_{t-\varepsilon}^{t}\int_{t-\varepsilon}^{t}\frac{N_0}{2}\delta(\tau-\lambda)]\, \mathrm d\tau \,\mathrm d\lambda\\ &= \int_{t-\varepsilon}^{t}\frac{N_0}{2}\,\mathrm d\lambda\\ &= \frac{N_0}{2}\varepsilon \end{align} If $N(t)$ is continuous-time white Gaussian noise with power spectral density $\frac{N_0}{2}$ and the samples are spaced at least $\varepsilon$ apart, say the $k$-th sample is at time $kT$ where $T\geq \varepsilon$, then we have that the $N[k]$ are independent zero-mean random variables with variance $\frac{N_0}{2}\varepsilon$, that is, _discrete-time white noise. If $N(t)$ is a Gaussian process, then the discrete-time process is a Gaussian process in which the random variables are independent $\mathcal N(0,\frac{N_0}{2}\varepsilon$ random variables.


$\endgroup$
  • $\begingroup$ @Dilipe: That was beautiful and educational. I would just add the following: Assuming that brownian motion is viewed as white noise ( don't know if it's definitely viewed that way in DSP ) there is some implied and possible not useful (in DSP) connection between white noise and the gaussian pdf. This is because the three requirements for brownian motion, B_t, are known to be 1) B_t starts at zero so B_0 = 0. 2) stationary independent increments and 3) For every t >0, B_t is normal(0,t). Therefore, the white noise process, if viewed from its beginning, is a normally distributed process. $\endgroup$ – mark leeds Mar 16 '18 at 20:07
  • $\begingroup$ One other thing since I may not have been clear. My point above is that, in the ideal case, where one could sample brownian motion say every 1 second, then that sampled process could be viewed as N(0,1) in the discrete case. As I said, I don't know if brownian motion is viewed as white noise in DSP world. $\endgroup$ – mark leeds Mar 17 '18 at 0:07
  • $\begingroup$ @markleeds Brownian motion is the integral of white noise from $0$ to $t$, not white noise itself. I don’t know if people in the DSP world view Brownian motion as white noise but if they do, they shouldn’t $\endgroup$ – Dilip Sarwate Mar 17 '18 at 2:36
  • $\begingroup$ right. it's the integral. but, if you start at $t1$ and then stop at $t2$, then brownian motion over that period is N(0,t2-t1) due to independent increments. So, sampling each observation at 1 second, gives you a normal(0,1) which can be viewed not as an integral and just as a random variable. See Hamilton or Mikosh for how the FCLT can be used to derive brownian motion. $\endgroup$ – mark leeds Mar 17 '18 at 5:05
  • $\begingroup$ @Dilipe: It would be useful if you would provide your definition of white noise. we had a problem with correlation and covariance the other day so possibly something similar here. thanks. $\endgroup$ – mark leeds Mar 17 '18 at 5:08
0
$\begingroup$

Points to be noted:

  • White noise has zero mean. So, $\mu = 0$.
  • White noise also means two samples separated in time are uncorrelated.

$R_{X(t)}(\tau) = E[X(t)X(t+\tau)] = E[X(t)]E[X(t+\tau)] = 0$ when $\tau \neq 0$.

On the other hand, when $\tau = 0$ :

$R_{X(t)}(\tau) = E[X^2(t)] = \sigma^2 $ which is the variance of gaussian noise.

However, there's a catch. I couldn't provide an explanation for $\delta(\tau)$- notation for infinite amplitude at $\tau = 0.$

I think this $\delta(\tau)$ function can be accounted to the classical definition of Auto-correlation function where there is an integral over time as well.

There seems to be a relevant discussion here.

$\endgroup$
  • $\begingroup$ This answer is mostly incorrect. The most relevant part is the reference to MattL's answer to a related question at the very end of the answer. $\endgroup$ – Dilip Sarwate Mar 16 '18 at 16:28
  • $\begingroup$ @DilipSarwate Can you please elaborate what is wrong with the answer? $\endgroup$ – Manideep Mar 17 '18 at 1:49
  • $\begingroup$ @Manideep: If one is allowed to definite $R_{x((t)}(\tau)$ as the covariance of, $X(t)$ at lag $\tau$, where $X(t)$ is a brownian motion process, then there is nothing wrong what you said. Your definition is just a different way of expressing independent increments. But, I'm not from DSP world, so I'm not sure if it can be defined that way in DSP. framework. $\endgroup$ – mark leeds Mar 17 '18 at 4:58
  • $\begingroup$ @Manideep: I would take the * out of your definition though. That makes it look like convolution. $\endgroup$ – mark leeds Mar 17 '18 at 5:00
  • $\begingroup$ @markleeds Thanks for the suggestion. I did as you suggested. $\endgroup$ – Manideep Mar 17 '18 at 6:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.