Low-Pass Filtering on Small Chunks of Audio

Question

I've got a script (Python) that chops 44.1 kHz realtime audio into 10 ms chunks, processes each chunk, and stitches the chunks back together with no added artifacts. I'm trying to create a low-pass filter that operates on each chunk, but every filter I've tried introduces unwanted artifacts.

I've tried creating various types filters, including IIR (Butterworth) and FIR (Hann, Hamming, rectangular windows, etc.), and Remez. The attempted methods of applying the filters to the input signal include just a straight transfer function, convolution, and a forward-then-backward filter (namely scipy.signal.filtfilt). I've also tried the moving average approach, but I don't know what to do near the endpoints of the chunk.

Is there such a filter that is designed for this kind of application?

INPUT WAVE (lots of harmonic content)

|<-- 1 CHUNK  -->|
:                :                :                :
:                :                :                :
: -----       ---:-       -----   :   -----       -:
:|     |     |   : |     |     |  :  |     |     | :
:|     |     |   : |     |     |  :  |     |     | :
:|     |     |   : |     |     |  :  |     |     | :
:       -----    :  -----       --:--       -----  :
:                :                :                :
        V                V                 V
        V                V                 V
     FILTER            FILTER            FILTER
        V                V                 V
        V                V                 V
:                :                :                :
:   ---         -:-         ---   :     ---        :
:  /   \       / : \       /   \  :    /   \       :
: /     \     /  :  \     /     \ :   /     \     /:
:/       \   /   :   \   /       \:  /       \   / :
:         ---    :    ---         :--         ---  :
:                :                :                :

OUTPUT WAVE (less harmonic content)

EDIT: Following the answer from @hotpaw2, I wish to try the IIR approach first (I'm a novice at this, but my gut tells me try IIR first). For discussion purposes, let's define these arrays:

B = transfer function numerator coefficients
A = transfer function denominator coefficients
I0 = previous input array (same size)
I1 = current input array (441 samples)
R0 = previous output, post-filtered (DO WE NEED?)
R1 = current output, desired post-filtered result array

Do I generate the initial conditions from I0 only, or do I need to figure R0 into the equation? To ask another way, here's the Python code I'm trying:

initCon = scipy.signal.filtic(B, A, I0)

Once I have this set of initial conditions, are they directly applied to the filter, or is there more that I need to do with them? Again, the Python code:

R1, _ = scipy.signal.lfilter(B, A, I1, z = initCon)

As the code stands now, it executes but there is a lot of off-frequency buzzing.

EDIT 2: Just thought I'd post the output of the filter with a sine wave input. It's quite obvious something goes wrong every 441 samples (the beginning of each chunk):

Answer 1

Filtering a data block usually results in more data than fits in the size of the block. If you throw away this added data, that will produce artifacts across blocks.

For FIR filters, you need to pad each chunk or block with at least the length of the impulse response of your filter before filtering (>= N+M-1). Then use overlap-add or overlap-save (FFT fast convolution algorithms) to process all the extra results from convolution of the added padding (usually carry them over to the next chunk or chunks). Otherwise you are either getting circular convolution artifacts in each block, or are throwing away part of the filter response from applying your filter to each block.

For IIR filters, you need to carry over the filters internal state between chunks, and not start over from zero or other default initial state of the IIR. That state still carries energy from the filter result, and if you throw it away, you will get glitches between chunks.

Answer 2

This post solved my problem: https://stackoverflow.com/questions/58014131/implementing-blockwise-low-pass-iir-filter-in-python

In the post, Warren Weckesser directed me to his own paper on the topic, where I read that an SOS filter is the solution I need, if I'm going to use an IIR filter. The same paper had a sample of Python code which helped me understand how to handle the initial conditions properly.

Answer 3

This works, but I'm not sure why! The initial conditions are the result of multiplying Z (I don't understand Z's purpose, and why lfiltic() didn't work at all) with the last couple of values from the previous result. Note those last two elements are in reverse order, which maybe says something.

Bparam, Aparam = signal.iirfilter(2, 0.020, btype = 'lowpass', analog =
       False, ftype = 'butter')       # 2nd order Butterworth coefficients

Z = signal.lfilter_zi(Bparam, Aparam) # Part of the init conditions calc

IC = Z * (prevSignal[::-1])[0:2]      # Reverse prevSignal and then grab
                                      #   only the last two elements

filteredSignal, _ = signal.lfilter(Bparam, Aparam, inputSignal, zi = IC)
                                      # Result is continuous and clear

prevSignal = filteredSignal           # Save for the next pass

However, This only works for the 2nd order lowpass filter. If I change the order (e.g. 4th order), the artifacts reappear conspicuously. Is there perhaps a more general form of this that actually works with higher orders of filters?