Invertible low-pass (Butterworth) filter in python (scipy)?

Question

Question

Is there an invertible low-pass filter built into scipy.signal (or other python package)? If so, what is it? If not, why not (is there something particularly difficult about inverting a low-pass filter)?

Elaboration

I need an invertible (digital, first-order, for concreteness) low-pass filter, such as a butterworth filter. I need to apply the inverse of it to precompensate an analog signal with $(x,y) = (\mathrm{time}, \mathrm{voltage})$. The signal is converted from digital to analog after filtering.

Work done so far

Naively, (as far as I understand) I can do the following:

import numpy as np 
import scipy
# define signal here
num_order = 1
cutoff_frequency = 1e4 # Hz 
time_constant = 1/(2*np.pi*cutoff_frequency) 
lpf_b, lpf_a = scipy.signal.butter(num_order, btype='low', Wn=1/time_constant/(2*np.pi*sample_rate/2))
precompensated_signal = scipy.signal.lfilter(lpf_a, lpf_b, signal)

where an example signal is defined, as (e.g.)

sample_rate = 1e6
t_period = 2*1e-3
num_samples = int(round(t_period*sample_rate))
amplitude = 0.23
signal = amplitude*np.ones(num_samples)

This approach works when inverting a high-pass filter.

Applying the filter in the forward-direction (filtered_signal)

filtered_signal = scipy.signal.lfilter(lpf_b, lpf_a, signal)

produces the expected signal (check by running the code or see image):

But applying it backward (precompensated_signal) to precompensate yields an oscillating signal:

Looking at the coefficients, I find

lpf_a = [ 1.0, -0.93906251]
lpf_b = [0.03046875, 0.03046875]

Seemingly having two identical coefficients in b as the first and only coefficients is what renders the filter non-invertible.

There is a related question and answer from 2016. Based on the answer, taking the filter coefficients given by scipy.signal.butter and modifying them as follows

lpf_b, lpf_a = scipy.signal.butter(num_order, btype='low', Wn=1/time_constant/(2*np.pi*sample_rate/2))
lpf_b_2 = [1 + lpf_a[1]] # note that lpf_b_2[0] = lpf_b[0] + lpf_b[1] 
filtered_signal_2 = scipy.signal.lfilter(lpf_b_2, lpf_a, signal)
precompensated_signal_2 = scipy.signal.lfilter(lpf_a, lpf_b_2, signal)

yields a filter which behaves identically to the original one in the forward direction (filtered_signal_2):

and behaves a little better in the backward direction (precompensated_signal_2):

though the resulting signal is still questionable. The only coefficient that differs from amplitude is the first one (and this seems to be true even when changing the cutoff frequency), and worryingly then the voltage at the first datapoint gets very large. This feature seems to be true for the filter in the above linked answer.

Edit

ZR Han suggested (thanks!) to shift the pole of the filter. As far as I understand, it can be done as follows:

z, p, k = scipy.signal.tf2zpk(lpf_b, lpf_a)
# the zero z = [-1.0] as he suggested
z = [-0.95]
lpf_b, lpf_a = scipy.signal.zpk2tf(z, p, k)

This results in

lpf_b = [0.03046875, 0.02894531]
lpf_a = [ 1.0, -0.93906251]

But unfortunately this did not remove all of the oscillation, as can be seen by plotting the precompensated signal (scipy.signal.lfilter(lpf_a, lpf_b, signal) with the modified coeffs):

Perhaps this is caused by a poor choice in z/my lack of formal understanding of what is "a small shift". Since the precompensated signal is converted to analog, ill behaviour is not tolerated in the solution unfortunately.

Directions to go from here

1. Explaining what the modified filter corresponds to
2. Finding the filter from the answer from scipy
3. Finding another filter which is better suited to be used with signal that is converted to analog (smooth and finite)

Context: I have no formal background in digital signal processing, but I can learn what I need (faster if I'm pointed to a source).

Answer 1

Is there an invertible low-pass filter

No

is there something particularly difficult about inverting a low-pass filter?

Yes. Digital low pass filters (in the most common sense) have a zero at Nyquist which means that the inverse has infinite gain at Nyquist and is unstable.

Seemingly having two identical coefficients in b as the first and only coefficients is what renders the filter non-invertible.

Yes. That creates the zero at Nyquist.

a filter which behaves identically to the original one in the forward direction

It does not. Changing the "b" coefficient to a single one will move the pole away from the unit circle on the real axis which changes the high frequency behavior. The signal only looks the same since it has very little high frequency content

Depending on you application you are probably better off with complimentary shelf filters. A low-pass filter destroys information which is unrecoverable. In practice if any filter has a lot of attenuation the inverse will have a lot of gain which is problematic in terms of noise performance.

Context: I have no formal background in digital signal processing, but I can learn what I need (faster if I'm pointed to a source).

Digital Signal Processing is fairly math heavy and it really helps to have some idea about the theoretical foundations. A good starter could be:

https://ccrma.stanford.edu/~jos/filters/

Answer 2

Following up the suggestion to use so-called shelving filters by Hilmar (thanks!), with the biquad_cookbook module that endolith linked (thanks!). The package (written in 2013) runs with (conda) python 3.8.10 out of the box.

Mini example:

import numpy as np
import scipy
import biquad_cookbook # need the file in the working directory

precompensated_signals = []
precompensated_and_filtered_signals = []
for g in [3, 6, 9, 12]:
    shelf_b, shelf_a = biquad_cookbook.shelf(Wn=1/time_constant/(2*np.pi*sample_rate/2), 
        dBgain=g, S=1, btype='high', ftype='half', analog=False, output='ba')
    precompensated_signals.append(scipy.signal.lfilter(shelf_b, shelf_a, signal))
    precompensated_and_filtered_signals.append(scipy.signal.lfilter(lpf_b, lpf_a, 
        scipy.signal.lfilter(shelf_b, shelf_a, signal)))

where lpf_a and lpf_b are defined as in the question, to yield the smooth, finite, albeit slightly oscillating precompensated signals (precompensated_signals; note that we applied the shelf filter in the forward direction - the filter is directly the precompensating filter):

Taking these through the original low-pass filter (precompensated_and_filtered_signals modelled as the first order butterworth filter from the question) yields the following signals:

So, this is a partial solution. For example, at 12 dB gain, the signal reaches the desired amplitude in roughly tenth of the time compared with the non-precompensated signal. The tradeoff is that the amplitude will overshoot due to the ripple in the precompensated signal (created by the filter).