I am generating a time-stream data set that needs to be filtered. The data-set is pretty large (i.e. too large to be filtered in one go). I'm trying to split the filtering operation up into chunks but I obviously get a spike at each chunk start. Is there a correct way to split a large data set up into smaller ones and have a continuous result - i.e (pseudcode).

timestream # variable with say 1e9 data points
b,a = sig.butter(1, 0.16, 'lowpass')

Split the timestream up in to say 1000 "chunks"

f_t = []
for c in chunks:
   t_data = sig.lfilter(b,a,c)
   f_t.append(t_data)