# Envelope reconstruction algorithm for audio processing

I need to reconstruct the envelope of a sound.

Audio data are professionally-recorded natural sounds (speech, bird songs) with very little noise. I would prefer working in the time domain rather than in the frequency domain (I've seen some algorithms based on FFT transformations that looked overcomplicated for what I need). The algorithm will be implemented in an interpreted language so it needs to stay "light" in computation.

As a first approach, I considered using a peak detection algorithm, then doing a linear interpolation between the peaks. But isn't there some pitfalls with such a naive approach? Are there some standard ways of implementing envelope reconstruction in the time domain that would better suit my needs?

FWIW, I'm not familiar with digital signal processing vocabulary, so do not hesitate to reword my question if I misused some terms

• Have read about the hilbert transform, and/or analytic signal? If you wanted to do it ‘correctly’ that’s what I’d recommend. – Dan Szabo Nov 25 '19 at 14:48
• @Dan Thanks for the comment. I heard about the Hilbert transform, but I'm not sure to fully understand what I'm reading here and there. Esp. is it suitable if there is not a single frequency carrier? In my use case, a sound can have an attack dominated by some frequencies, then shifting to a different coloration for the sustain (see teachmeaudio.com/recording/sound-reproduction/sound-envelopes) – Sylvain Leroux Nov 25 '19 at 14:54
• A Hilbert Transform should be well suited for such an application. It would be worthwhile to get set up in python to run some trials if you are so inclined. There are practical limitations to hilbert transforms, but I bet you’d get good results without a ton of work – Dan Szabo Nov 26 '19 at 2:29
• The standard way to get the envelope of a signal is to pass it through a lowpass filter. FIR filters are easy to implement in the time domain. – dsp_user Nov 26 '19 at 8:35
• Thanks, @dsp_user. I've got some good advice in the comments. Would you consider posting that as an answer? – Sylvain Leroux Nov 26 '19 at 10:51

As I said in a comment, you can get the envelope of a signal by running it through a lowpass filter.

The steps required for this (usually) are

1. Go through all the samples (x(N) ) and check for negative samples. Convert them to positive values

2. Implement a lowpass filter (FIR) by creating a filter kernel of appropriate length M (h(M) ). Note that your FIR filter (sin(x)/x) should normally be multiplied by a window function (e.g Hamming, Blackman etc).

3. Convolve your signal with the filter kernel.

for(int i=0;i<N;i++)

for(int j=0;j<M;j++)

y[i+j] = y[i+j]+x[i]*h(j);


Note that the output (filtered) signal is usually right shifted by M samples. Also, M is often chosen to be odd in order to create a perfectly symmetrical filter although I'm sure not all implementations adhere to this.

• Thank you for having posted that answer @dsp. FWIW, since you original comment, I also experimented with moving average (which, if I understand it correctly, is a kind of FIR) but it was not quite satisfactory. I will try your solution to see if it performs better. – Sylvain Leroux Nov 26 '19 at 13:16
• "the output (filtered) signal is usually right shifted by M samples." This shouldn't be an issue since I'm not working in real-time, so I would be able to compensate for that. – Sylvain Leroux Nov 26 '19 at 13:18
• @Sylvain Leroux, Yes, a moving average can work well for some simple cases but a lowpass windowed sinc is much better. If you need help with step #2, I can provide some code. – dsp_user Nov 26 '19 at 13:25
• Thanks for your offer @dsp! The implementation per se will not be an issue. It is trickier for me to choose the "right" filter and windowing functions, though. – Sylvain Leroux Nov 26 '19 at 13:38

I would recommend a streaming RMS detector. The standard approach for computing a streaming RMS detector is to square the input samples and then apply these to a 1st-order lowpass filter. If you want the output in dB, take 10*log10() of this quantity. If you want the output in volts, take the square root of this quantity. If Logs and square-roots are too heavy for your processor, there are low-mips ways to approximate them. Note that RMS detection offers a good compromise between a peak detector (which is often over-responsive to very narrow transients) and average (which is sometimes too slow). Also, the ear tends to approximate RMS dynamics in its perception of loudness.