Rolling sum of dataframe with custom function gets really slow on large averaging windows. Can I fix with convolve?

Question

I am using the following function to estimate the Gaussian window rolling average of my timeseries. Though it works great from small size averaging windows, it crushes (or gets extremely slow) for larger averaging windows.

def norm_factor_Gauss_window(s, dt):
    
    numer         = np.arange(-3*s, 3*s+dt, dt)
    multiplic_fac = np.exp(-(numer)**2/(2*s**2))
    norm_factor   = np.sum(multiplic_fac)
    window        = len(multiplic_fac)
    
    return window,  multiplic_fac, norm_factor

# Create dataframe for MRE
aa = np.sin(np.linspace(0,2*np.pi,1000000))+0.15*np.random.rand(1000000)
df = pd.DataFrame({'x':aa})

hmany  = 10
dt     = 1      # ['seconds']
s      = hmany*dt  # Define averaging window size ['s']

# Estimate multip factor, normalizatoon factor etc
window, multiplic_fac, norm_factor= norm_factor_Gauss_window(s, dt)

# averaged timeseries
res2 =(1/norm_factor)*df.x.rolling(window, center=True).apply(lambda x: (x * multiplic_fac).sum(), raw=True, engine='numba', engine_kwargs= {'nopython': True,  'parallel': True} , args=None, kwargs=None)

#Plot
plt.plot(df.x[0:2000])
plt.plot(res2[0:2000])

I am aware that people usually speed up moving average operations using convolve(e.g., https://stackoverflow.com/questions/14313510/how-to-calculate-rolling-moving-average-using-python-numpy-scipy)

Would it be possible to use convolve here somehow to fix this issue? Also, are there any other suggestion that would help me speed up the operation for large averaging windows?

Answer 1

With the help of @Hilmar an @Marcus I was able to drastically improve the speed of this code using the following:

from scipy import signal
def norm_factor_Gauss_window(s, dt):
    
    numer         = np.arange(-3*s, 3*s+dt, dt)
    multiplic_fac = np.exp(-(numer)**2/(2*s**2))
    norm_factor   = np.sum(multiplic_fac)
    window        = len(multiplic_fac)
    
    return window,  multiplic_fac, norm_factor

# Create dataframe for MRE
aa = np.sin(np.linspace(0,2*np.pi,1000000))+0.15*np.random.rand(1000000)
df = pd.DataFrame({'x':aa})

hmany  = 10
dt     = 1      # ['seconds']
s      = hmany*dt  # Define averaging window size ['s']

# Estimate multip factor, normalizatoon factor etc
window, multiplic_fac, norm_factor= norm_factor_Gauss_window(s, dt)

# averaged timeseries

res2 = (1/norm_factor)*signal.fftconvolve(df.x.values, multiplic_fac[::-1], 'same')

#Plot
plt.plot(df.x[0:2000])
plt.plot(res2[0:2000])