How can I get the used Hilbert filter coefficients from the Matlab function envelope()?
filter_length = 10;
[up1,lo1] = envelope(input_data, filter_length,'analytic');
From the mathworks documentation of the function envelope():
The filter is created by windowing an ideal brick-wall filter with a Kaiser window of length
fl
and shape parameter $\beta = 8$.
So without hacking the function you can't directly get the filter coefficients, but you can easily find them yourself by just doing what they do, i.e., windowing the ideal impulse response by a Kaiser window. The ideal impulse response of a Hilbert transformer is given by
$$h[n]=\begin{cases}\frac{2}{n\pi},& n\textrm{ odd}\\0,&n\textrm{ even}\end{cases}\tag{1}$$
You just have multiply these coefficients with the window of the desired length.