4
$\begingroup$

Given a gaussian white noise with mean 0 and $\sigma = 1$. I'm interested in the PSD (power spectral density), when the signal is plain up- or downsampled. No interpolation, no filtering nothing.

There are some related topics in the web, but they are always ending with someone considering low- or passband filters. They are also often speaking about the aliases which get e.g. back in the nyquist band when downsampling, but they never properly describe it with math:

So, what happens with the given PSD, (which should have an amplitude of 1/fs between -fs/2 and fs/2) when it's led trough a down and upsampler?

I assume first, up and down or the opposite should result in the same PSD. Otherwise one could just up and downsample to benefit from some noise reduction. I could also imagine that up and down sampling make no changes at all to white noise. On the other hand, there might be an effect when downsampling, due to aliases.

$\endgroup$
  • $\begingroup$ Hi Mr. Shannon! All the signals are discrete-time seqeunces right? So you are referring to discrete-time up / down sampling. But without filtering, these operations are more conveniently defined as expansion and compression... $\endgroup$ – Fat32 Aug 29 '18 at 19:21
  • $\begingroup$ Yes exactly! Heard that too. Still not convinced which are used most often. Every book I read so far used other terms -.- $\endgroup$ – Mr.Sh4nnon Aug 29 '18 at 19:26
  • $\begingroup$ Please clarify what you mean by up and down sampling with no filtering. From that statement I would say the upsampling is a zero insert (as it should be with no filtering) and the down sampling is a decmiation (only passing every Dth sample for decimate by D). Is this what you mean? The filtering is what properly removes the aliases to complete the decimation and interpolation operation but let's be clear on what you mean in your case. $\endgroup$ – Dan Boschen Aug 29 '18 at 23:35
  • $\begingroup$ Thank you for your answer! Sorry I assumed it was known by the definition of down and upsamling. Yes Im speaking about insertif zeros or taking just every n-th sample. $\endgroup$ – Mr.Sh4nnon Aug 30 '18 at 5:36
  • $\begingroup$ Hi mr.Shannon. So you've got your answer. Which shows the relation between the PSDs of input and output after the expansion and comression operations. According to what I've derived (assuming no mistake), for the case of expansion operation, the output is not WSS, hence it does not have a PSD. See if this suffices for you. $\endgroup$ – Fat32 Sep 2 '18 at 10:54
3
$\begingroup$

Let me try to explain what happens to the PSD (power spectral density) of a discrete-time Gaussian white noise $x[n]$ of power $\sigma_x^2$ after it's either expanded by $L$ (insertion of L-1 zeros between consequitive samples of $x[n]$), or compressed by $M$ (aka decimation, selecting every M-th sample of $x[n]$).

First, let's put a convenient framework, by defining required quantities and their assumptions.

A discrete-time random process (RP) is constructed by a set of random variables (RV) and represented by $\{X[n,s)\}$ where integer $n$ denotes time-indexing for each RV and $s$ denotes outcome of a random experiment associated with those RVs or the RP. Customarily, an outcome $s$ selects a complete discrete-time waveform $x[n]$, which is an instance of the RP, from the ensamble set of it. The ensamble set includes all possible discrete-time waveforms. For notational simplicity, the sequence $x[n]$ is referred to as the random process, unless stated otherwise.

Probabilistic analysis of the RP $x[n]$ proceeds by defining various joint PDFs (probability density/distrbution functions) between random variables of the RP. Among those joint PDFs, (and moments associated) two of them become most useful and important for practical engineering applications. They're the mean and the Auto-Correlation Sequence (ACS) of the process, as defined below for a real $x[n]$ :

1- Mean of a RP is: $\mu_x[n] = \mathcal{E}\{x[n]\} $

2- ACS of a RP is: $\phi_{xx}[n,m] = \mathcal{E}\{x[n]x[n+m]\}$

As can be seen, in general the mean and ACS of a RP depend on time index $n$. The ACS depending also on the lag m. A very important simplification for the analysis of RP is made by defining what's known as a wide sense stationary (WSS) RP, whose mean and ACS are defined to be independent of time; i.e.,

1- Mean $\mu_x[n] = \mu_x = \mathcal{E}\{x[n]\} $

2- ACS $\phi_{xx}[n,m] = \phi_{xx}[m] = \mathcal{E}\{x[n]x[n+m]\}$

Now, for given a WSS random process $x[n]$, its ACS $\phi_{xx}[m]$ holds a very important position. The power spectral density (PSD), $S_{xx}(e^{\omega})$, of a WSS RP is defined to be the discrete-time Fourier transform (DTFT) of its ACS:

$$ S_{xx}(e^{\omega}) = DTFT \{ \phi_{xx}[m] \} = \sum_{m=-\infty}^{\infty} \phi_{xx}[m] e^{-j\omega m} $$

At this point, it's clear that the PSD of a RP exists iff the RP is WSS and further that its ACS is stable so that its DTFT converges. I assume @DilipSarwate may comment if a (major) ill-statement was presented up to here.

we are now ready to tackle your problem !

The compression by integer M (which you refer to as downsampling without filtering) operation is:

$$ x[n] \longrightarrow \boxed{ M \downarrow} \longrightarrow y[n] ; ~~~y=x[Mn] $$

Given an i.i.d (independent, identically distributed) zero mean, WSS, (Gaussian) white noise RP $x[n]$ with power $\sigma_x^2$ it can be shown that its ACS and PSD are:

$$\phi_{xx}[m] = \sigma_{x}^2 \delta[m]$$ $$S_{xx}(e^{j\omega}) = \sigma_x^2 ~~~, ~~~\text{ for all } \omega $$

Then what is the PSD associated with the compressed output $y[n]$ ? To show this, first we must find its ACS and see that the RP $y[n]$ is indeed WSS.

$$\phi_{yy}[n,m] = \mathcal{E}\{y[n]y[n+m]\} = \mathcal{E}\{x[Mn]x[M(n+m)]\} = \mathcal{E}\{x[Mn]x[Mn+ Mm)]\} = \phi_{xx}[Mm] = \sigma_x^2 \delta[Mm] = \sigma_x^2\delta[m]$$

Hence we see (with a slightly bold step of treating $Mn$ the same as $n$ alone, which's justified by the WSS and whiteness of the input $x[n]$) that the ACS associated with the compressed signal $y[n]$ is identical to the ACS of the input $x[n]$. Hence their PSD's are also the same:

$$S_{yy}(e^{j\omega}) = \sigma_x^2 ~~~,~~~\text{ for all } \omega $$

Next we apply the same for the expansion (you refer upsampling) process which can be shown to be as: $$ x[n] \longrightarrow \boxed{ L \uparrow} \longrightarrow y[n] = \begin{cases} { x[n/L] ,~~~n=rL \\ 0 ,\text{ otherwise} }\end{cases} $$

Let's try to compute the ACS of the expanded output $y[n]$. $$\phi_{yy}[n,m] = \mathcal{E}\{y[n]y[n+m]\} = \begin{cases} { \sigma_x^2 ,~~~ m=0 ~~~\text{ and } n=rL \\ 0 ,~~ m \neq 0~~~\text{ or } n \neq rL }\end{cases} $$

When computing $\phi_{yy}[n,m]$, I assumed that the stuffed zeros were deterministic random variables with a pdf of $f_X(x)=\delta(x)$, which can only take a value of $0$ with a probability of one. They are not independent in themselves but independent from nonzero samples. Furthermore, since their value is always zero, when multiplied inside an expectation operator they always yield a zero result.

Now there is a problem! Eventhough the input $x[n]$ is WSS, the output is not. As its ACS $\phi_{yy}[n,m]$ depends not only on the lag $m$, but also on the exact time instance $n$. Hence its ACS is not of the form that's suitable for the DTFT expression. So at this point we can say that the PSD associated with $y[n]$ does not exist in the form of $$S_{yy}(e^{j \omega}) = DTFT\{ \phi_{yy}[m] \} $$

May be a parametric Fourier transform or a Short-Time Fourier transform (STFT) or a 2D Fourier transform can be utilized but I'm not very sure about it.

Finally, we can state the following observation that since the expansion and compression operators are not LTI systems (they are time varying), then we are not surprised to see that their outputs are not WSS even if their inputs are so, which would not be possible with LTI systems.

$\endgroup$
  • $\begingroup$ Very good answer Fat32! So in the second case it would be cyclo-stationary, can you expand your answer to include that and how the final variance is then related to the input variance? Also since the equations are rigorous, the variance is scaled by N where N is the number of samples in the FFT so that $S_xx(e^{j\omega}) = \frac{1}{N} \sigma_x^2$. If you agree, perhaps cleaner to handle that in your first definition relating the ACS to the DTFT (just to be complete). $\endgroup$ – Dan Boschen Sep 2 '18 at 12:45
  • $\begingroup$ (So in the case of Interpolation the power spectral density does not exist since the process is not stationary, but the average power spectral density DOES exist since the process is cyclo-stationary, correct?) $\endgroup$ – Dan Boschen Sep 2 '18 at 12:59
  • $\begingroup$ Yes it's cyclo-stationary in the sense that its autocorrelation sequence $\phi_{yy}[n,m]$ is repeating in the time index n. But I decided to cut it there, by simply indicating that process is not WSS. A little easy (lazy) approach. Probbaly a further analysis could be added. Note that output variance is time varying as well (in a cyclic manner). So let's see if I can add further. Thanks for the comments. $\endgroup$ – Fat32 Sep 2 '18 at 13:10
  • $\begingroup$ Yes good point, a time varying variance so we then must have an average variance like we have an average power spectral density to be the units of interest in that case. $\endgroup$ – Dan Boschen Sep 2 '18 at 13:45
2
$\begingroup$

I believe that Gaussian White Noise (or any white noise) will be white with the same (variance) before and after down sampling (if no filtering is applied). For interpolation the variance goes down by the rate of the interpolation and the spectrum, although constant over the digital frequency range, is no longer independent over all samples in the primary Nyquist zone of the new interpolated signal (so is this still "white", I am not sure??).

This is very easy to see in the case of decimation with consideration to the figure below, which shows an AWGN (Additive White Gaussian Noise) discrete time process in time and it's histogram to the left. The standard deviation is observable as proportional to the width of the Gaussian bell curve shown in the histogram. Gaussian refers to the distribution of the amplitudes, and white refers to point that EVERY sample will be independent of the next. This means in frequency the power spectral density is flat across the entire digital frequency space (+/-$F_s/2$ where $F_s$ is the sampling rate). That said, since every sample is independent of the other, it would not make any difference to this plot if we took every other sample (and extended the span twice as long to collect the same number of datapoints); we would end up with the same result, the same histogram and therefore the same standard deviation. We can continue this argument for every 10th sample, every 100th sample etc. What is happening here is the total energy in the signal is folded into the new lower Nyquist zone after decimating such that the total energy is conserved (this is consistent with the treatment of aliases etc through the decimation process).

AWGN

I do not have as clean of an answer yet for the case of interpolation; for interpolation we insert zeros, which is convolving in the time domain with an impulse, so replicating our frequency spectrum but creating images where the previous $F_s$ existed and every multiple of the previous $F_s$, such as shown in the figures below for an interpolate by 4 (so the $F_s/2$ that is shown is the new $F_s$, so an image of the original spectrum exists at $F_s/4$, $F_s/2$, $3Fs/4$ and $F_s$ in this case).

Zero insert

Interpolation spectrum

So we see that when the original spectrum is white, the passbands shown would extend to the boundary such that the new spectrum would also be white (!White only in the sense that the PSD is constant, but is this indeed still "white" in a strict sense???? Read on and see the plots as the spectrum is replicated and therefore each frequency sample is not independent of the other samples, which is quite different from an AWGN process). It is also easy to see mathematically that the variance in this case goes down by N where N is the interpolation factor as we are increasing the total number of points but adding zeros to the mean square summation in computing the variance. However, at this point I am puzzled as the spectrum has a constant PSD for all frequencies, but in time each sample is clearly not independent of the next in this case as we have several zero samples in a row. We might note that although the spectrum is fully occupied evenly, it itself is not completely random and independent from sample to sample but replicated in every expanded Nyquist zone. So white in frequency coinciding with independence from sample to sample in time must also imply independence from sample to sample in frequency?

Below is an FFT for an example AWGN process before and after a 10 sample zero insert. The standard deviation went down by $\sqrt{10}$ as expected but also note the replication of the spectral pattern (also as expected).

Example FFT of AWGN process (the y-axis is the magnitude in dB scale and the x-axis is the FFT bin number):

AWGN

FFT of same process after interpolate by 10 with zero-insert (the y-axis is the magnitude in dB scale and the x-axis is the FFT bin number):

AWGN with interp by 10

So the result will be Gaussian distributed with a big impulse in the distribution at zero, and will have a constant power spectral density. My remaining confusion and point here is if we can still call that White Noise (Certainly it is not Gaussian due to the impulse, but is the definition of white noise simply a constant power spectral density? If so, we have an example here where not all samples in time are independent). My understanding up until going through this is a white spectrum and independence from sample to sample in time went hand-in-hand. (I may ask this as a separate question but keeping it here for now in case someone else can clear up this remaining point of confusion).

$\endgroup$
  • $\begingroup$ Dan, i think you hit it squarely regarding upsampling. as i understand upsampling, the gap between the previous Nyquist and the new Nyquist (which is higher if we're upsampling) ideally is zero, not the same constant non-zero magnitude as below the previous Nyquist. $\endgroup$ – robert bristow-johnson Sep 1 '18 at 3:30
  • 1
    $\begingroup$ it's a replicated spectrum. not that of white noise. this is why your upsampling operation is incomplete. zero-stuffing is not the whole of upsampling. $\endgroup$ – robert bristow-johnson Sep 1 '18 at 4:12
  • 1
    $\begingroup$ @Fat32 Yes! I am reading it now at this very moment with a full cup of coffee. I got through the first half and it is excellent. Wish I could give it two +1s! $\endgroup$ – Dan Boschen Sep 2 '18 at 12:36
  • 1
    $\begingroup$ But I enjoyed very much the very thorough review you provided of the rigorous prob theory, helps keep that fresh in my head. $\endgroup$ – Dan Boschen Sep 2 '18 at 12:51
  • 1
    $\begingroup$ Btw. It's helpful to indicate that when filtering is included, the output of the downsampler is still white (iid uncorrelated samples) but the output of the interpolator is not white and is indeed correlated and bandlimited noise. $\endgroup$ – Fat32 Sep 2 '18 at 13:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.