# Does the error in the impulse response accumulate when applying a step-input?

Consider the model of the form $$y[n]=0.9y[n-1]+x[n]+e[n]$$, I forgot if this is called an ARX model but who cares, anyways assume $$e[n]$$ is a zero-mean white noise. I am applying a non-parametric system ID method to estimate $$h[n]$$ by applying a step input with magnitude $$B$$, then sit back and watch the output and afterwards I apply a shifted step input $$u[n-1]$$ with same magnitude $$B$$ and again watch the output. Let's say the output is $$y_{0}[n]$$, then really what I want is to difference $$y_{0}[n]-y_{0}[n-1]$$ because it will force the input to difference as well (because $$h$$ is time-invariant) and create for me a $$\delta[n]$$ a unit-impulse response.

Now my derivations are presented below: $$y_{0}[n]=0.9y_{0}[n-1]+Bu[n]+e[n]$$ Then I will apply $$z-$$transform but I want to abuse notation and represent $$z-$$transform in time domain using Ljung's abusive notation of using $$q$$ as a time-index that tells me I am representing $$z-$$transform in time domain : $$y_{0}[n](1-0.9q^{-1})=Bu[n]+e[n]$$ $$y_{0}[n]=\frac{1}{1-0.9q^{-1}}Bu[n]+\frac{1}{1-0.9q^{-1}}e[n]$$ Now If I shift $$y_{0}[n]$$ by $$1$$ to get $$y_{0}[n-1]$$ and difference it with the equation above I should force $$u[n]-u[n-1]$$ to give me a delta and I am also differencing the error $$e[n]-e[n-1]$$ which in the $$z-$$domain translates to passing $$e[n]$$ to a high-pass filter. $$y_{0}[n]-y_{0}[n-1]=\underbrace{\frac{1}{1-0.9q^{-1}}}_{\text{Filter}}B\delta[n]+\frac{1}{1-0.9q^{-1}}\underbrace{(1-q^{-1})}_{\text{HPF}}e[n]\tag{1}$$ Now I can write $$y_{0}[n]-y_{0}[n-1]=Bh[n]+c[n]$$ where $$c[n]$$ is a colored noise and if the signal-to-noise ratio is pretty high then I can say $$y_{0}[n]-y_{0}[n-1]\approx B\widetilde{h}[n]$$ where $$\widetilde{h}[n]$$ is the approximated $$h[n]$$, then I can write : $$\widetilde{h}[n]=\frac{1}{B}(y_{0}[n]-y_{0}[n-1])\implies \widetilde{h}[n]=h[n]+\frac{1}{B}c[n]$$

I plotted the step-response for $$B=2$$ and the impulse response for $$\lambda_{0}=0.05$$ and obtained:

While this yielded a good approximation of the step-input I received an ugly prediction for $$h[n]$$ why is it that noise is amplifying in the case of $$h[n]$$?

• I've just wanted to spell my opinions as I haven't analysed the theory in a comprehensive manner, but I've noticed that the two graphs are different in amplitude scales. I mean if you zoom the y-axis of the unit step response, you may get a similar view in terms of the noise amplitudes. But, again, I've just wanted to throw a comment which might be helpful. Commented Mar 25, 2022 at 11:13
• What do you mean by an ugly prediction ? Why is it ugly?
– Peter K.
Commented Mar 25, 2022 at 12:25

As written $$e[n]$$ does not pass through a high pass filter but is presented to the same input as $$Bu[n]$$. The block diagram and subsequent equation would appear as follows:

We see noise growth as we are adding accumulated noise from past samples to each new noise sample at the input. This structure is known as a "leaky accumulator" which has a low pass filter response when the feedback is positive and converges as long as the feedback coefficient is -1 < a < 1. The closer to 1, the tighter the bandwidth of the filter in frequency and the longer the impulse response will be in time (and the greater the noise accumulation).

The transfer function for this "leaky integrator" with the noise injected at the input is consistent with the OP's equation:

$$y[n]=0.9y[n-1]+x[n]+e[n]$$

For purposes of developing a transfer function, we consider the transfer function from each input to output separately, and given the linear system the resulting output will be the sum of the individual results for each input. I will refer to the signal input to output transfer function as $$H_1(z)$$ and the noise input to output transfer function as $$H_n(z)$$:

$$H_1(z)= Y(z)/X(z)$$ $$H_n(z) = Y(z)/E(z)$$

$$Y(z) = 0.9z^{-1}Y(z)+ X(z)$$

$$Y(z)(1-0.9z^{-1}) = X(z)$$

$$Y(z)/X(z)=H_1(z) = \frac{1}{1-0.9z^{-1}}$$

Similarly for the noise path in this simpler case with the noise at the input it is obvious we would get the exact same result:

$$H_n(z) = \frac{1}{1-0.9z^{-1}}$$

Below shows the frequency response as computed with freqz in Octave/Matlab:

The resulting output noise, with white noise at the input would be low pass filtered, with a 20 dB gain for the lower frequency components and approximately 6 dB attenuation at the highest frequencies. So we see error accumulation at the lower frequencies indeed, but noise reduction at the higher frequencies (the noise is low pass filtered just like the signal).

The step response with $$B=2$$, which is found from the inverse z transform of $$H(z)/(z-1)$$ (as the integral of the impulse response, or "unit sample response" for a discrete time system) is shown below (conveniently given by the control package in Octave/MATLAB:

And the impulse response (unit sample response) as the inverse Z transform of $$H(z)$$:

Note the Octave/MATLAB code for the above is simply:

sys = tf(1, [1 -.9], 1);
figure
step(2*sys)
figure
impulse(2*sys)

The frequency response in Octave/MATLAB is also quite simple:

freqz(1, [1 -.9])

Observe the results for a simulation with white noise alone at the input and output (we can add any other waveform and this same noise would just add to input and output):

In the simulation, the standard deviation of the noise at the input was 0.99135, and at the output it was 2.3209 demonstrating the noise growth overall. But importantly we must also consider the frequency characteristics, as we can see in the one-sided power spectral density plots below:

I don't know, if I am missing something here, but $$e[n]$$ and $$e[n-1]$$ are uncorrelated noise signals regarding the higher frequencies, but correlated regarding the low ones. By combining them, you will get a new noise signal with the same zero mean but a signal power that is higher by $$3\text{dB}$$ compared to either $$e[n]$$ or $$e[n-1]$$.

• Consider the case of simply adding e[n] + e[n-1] - The total noise power does increase by 3 dB (but will also no longer be white)-- the lowest frequency components of the noise will be correlated from sample to sample, and therefore have a 6 dB gain, while the highest frequency components at $fs/2$ will be nulled. Overall the gain is 3 dB but colored. Commented Mar 25, 2022 at 14:26
• Ditto for $e[n] - e[n-1]$. In that case the noise will be "blue" noise, with zero spectral density at zero frequency, rising to a peak at $\omega = \pi$. Commented Mar 25, 2022 at 15:05
• And double ditto for $e[n]-e[n-2]$. In this case the noise will be "green" noise (I made that up and never heard the term but makes sense to me) with zero spectral density at zero and $\omega = \pi$, rising to a peak at $\omega = \pi/2$- but still with a +3 dB increase in noise. Commented Mar 25, 2022 at 15:20
• But interesting to note in this case, the noise increase is NOT 3 dB but closer to 7.4 dB given the feedback structure instead of the feedforward structure-- we keep adding accumulated noise of past samples to the new noise samples coming in, which is why it is higher, but with a leaky accumulator so the output still converges in the long term. Commented Mar 25, 2022 at 15:26
• Yes, very interesting indeed :-)
– Max
Commented Mar 25, 2022 at 15:27