5
$\begingroup$

I am looking for a way to characterise the frequency response of the slope from a linear regression. We are exploring the effect of window length of the regression to the magnitude of the slope of the regression and find that for our data (discrete time series data), the slope is positively related to the window length.

The slope can be thought of as the expected change of our output variable for a given input variable over a time period equal to the window length. We find that for a short window length, this slope is very low whereas for a long window length the slope is high. We interpret this to mean that the system responds to changes in the input in the long term but not in the short term. To me, that sounds a lot like a low pass filter killing off the fast changes in the input but passing the slower (or rather, more sustained) changes.

The slope, or beta, of the regression for a rolling window is, in a way, similar to a moving average and it is possible to characterise the frequency response of a moving average filter as a function of its window length as shown here. I was wondering if anyone knew of a similar way to characterise the frequency response of the beta of a rolling regression as a function of its window length in a similar manner?

Note that I have read this paper which explores using regression beta as a filter but I was unable to get an intuitive understanding of why the larger the window length is, the lower the cutoff frequency appears to be.

Here are a few descriptive charts of our data that might show why we are asking this question:

enter image description here

In the chart above, the thick grey line is the regression over all the data (i.e. a long window length) and has a slope of 1.79. The coloured lines (and points) show the regressions for single years of data (the betas for these lines are in the brackets in the legend of the chart). The beta for any individual year is significantly lower than the overall beta. In other words the grey line is much steeper than the other lines. This is true for any arbitrary 1 year window on the data (i.e. there is no year-long window of data that has a slope as steep as the overall data).

This next chart shows the rolling betas for different window lengths:

enter image description here

So the question we are trying to answer (and hoping to do so by characterising regression as a LPF) is why do the lines in this chart have higher biases (higher on average) as the window length increases (e.g. the yellow line is higher than the green line). How can we interpret this phenomenon?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ This article could be helpful for you. $\endgroup$
    – Matt L.
    Aug 19, 2015 at 19:58
  • $\begingroup$ @MattL. thanks, I did link to that article in my question. Could you perhaps explain how or where the paper characterises the frequency response of the regression slope as a function of window length as I was not able to work this out from my reading of the paper (which I did not understand very well) $\endgroup$
    – Dan
    Aug 20, 2015 at 6:01
  • $\begingroup$ OK, I didn't notice that link. I'll have a look at that paper again as soon as I have the time, because it's been quite a while since I read it. $\endgroup$
    – Matt L.
    Aug 20, 2015 at 6:11
  • $\begingroup$ A better way to think about this would probably have been as a weaker form of simpson's paradox $\endgroup$
    – Dan
    Jan 11, 2021 at 12:43

3 Answers 3

Reset to default
0
$\begingroup$

Essentially, the beta is like a rolling average. A rolling average will lag the the time series, so can be very inaccurate if there is a drift.

A better way is to use exponential weightings (EWMA).

An even better way of is to use Kalman Filters.

You can find a description on how below:

https://www.quantopian.com/posts/quantopian-lecture-series-kalman-filters

Improve this answer
$\endgroup$
4
  • $\begingroup$ We are interested in the "physical" meaning of the beta / slope however but we'll look into those alternatives as well $\endgroup$
    – Dan
    Aug 19, 2015 at 13:51
  • $\begingroup$ The question does still stand though, how can we characterise the frequency response of linear regression? $\endgroup$
    – Dan
    Aug 19, 2015 at 13:58
  • $\begingroup$ @Dan. As can easily be seen the regresion slope forms a saw wave. Similarly the convolution would simply be B*X. Frequency response of a saw wave follows 1/f IIRC. However, regression line is infinite and not periodic, and as such doesn't really have a transform. But if you window it and pretend... Anyway, picking a shorter window moves the saw wave to a higher pitch and as such "rolls off" less highs. $\endgroup$
    – Dole
    Feb 16, 2016 at 23:10
  • $\begingroup$ @JPN The equation for a beta is cov(x,y)/var(y)... This is does not look like a moving average process, or linear process at all. The variance contains elements that squares the equation. $\endgroup$
    – Dole
    Feb 17, 2016 at 0:23
0
$\begingroup$

This is a statistics question and should not be asked here. But I will try to answer the question as best as I can (it won't be the best answer). First off, I don't see what the phenomenon you are observing has to do with frequency response or LPFs. Low pass filters makes data smoother, which is not what you have. Instead you have a situation, where estimate for one set of data differs greatly from a other set of data. The OLS estimator also is not an MA process, the formula being cov(x,y)/var(x). The filter described in the paper is not a linear time invariant filter.

Indeed, if you regress a line on white noise process or linearly growing process, the regression line should not change slope significantly as more data is gathered. Instead as the amount of data grows you should be more certain of that the coefficient is right, generating greater variance for the rolling process, but on average the slope should be right.

What you have here is not a process that satisfies Gauss Markov assumptions. As can easily be seen the expected value of the error is not zero, and there is heavy auto correlation. This means that the error term is correlated with the independent variable, producing a biased estimate. Besides that the data is not randomly selected.

Most importantly, the data is increasing as a function of time. This generates a so called spurious correlation. Since Y increases faster than X as time passes, the beta becomes upwardly biased. You can model autocorrelation with an MA process, but this is not really important...

I am sure statistics stackexchange (crossvalidated) would provide you with much more mathematically precise answer and a better explanation of what the phenomenon. One thing fix that might help is to difference both of the variables, which changes the autocorrelation structure of the error. This can be equally viewed as a (1-1) MA process applied to both of the variables, which is a comb filter at a high frequency, producing a high pass filter below the nyquist frequency. But the data has such a non-standard structure that I wouldn't be so sure that the fix will help you.

Improve this answer
$\endgroup$
0
$\begingroup$

See Torp 1997, Clutter rejection filters in color flow imaging: A theoretical approach, IEE Trans. Ultras., Ferro., and Freq. control.

Also see Hubbert et al. 2021, Using a Regression Ground Clutter Filter to Improve Weather Radar Signal Statistics: Theory and Simulations, JTECH

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.