Exponential averaging is performed using a first order IIR filter according to the difference equation:
\begin{equation}
y[n] = (1-\alpha)\;x[n]+\alpha \; y[n-1]
\end{equation}
where $x[n]$ and $y[n]$ are the input and output signals at time $n$ and the value of the single coefficient $\alpha$ is given by the equation:
\begin{equation}
\alpha = \exp( \frac{-1}{f_s \tau} )
\end{equation}
where $f_s$ is the sampling frequency and $\tau$ is the time constant.
In this light, it can be seen that exponential averaging filters are simply low-pass filters, the roll-off of which is -6dB/oct. The only free parameter is the cut-off frequency, which is specified by the time constant. Longer time constants will yield a lower cut-off frequency, as can be see in the Matlab code and plot below.
figure; hold all
fs = 8000;
tau = 10.^(-1:-1:-4);
alpha = exp(-1/fs./tau);
color = 'brgc';
for index = 1 : length( alpha )
[Hz fVec] = freqz( [1-alpha(index)], [1 -alpha(index)], 8192, fs );
plot( fVec, 20*log10(abs(Hz)), color(index), 'LineWidth', 2 );
legText{index} = sprintf( '$\\tau = 10^{%d}$', log10(tau(index)) );
end
hLeg = legend(legText,3);
set( hLeg, 'interpreter', 'latex' );
axis( [8000/8192 4000 -70 10] )
ylabel('Amplitude [dB]')
xlabel('Frequency [Hz]')
set( gca, 'xscale', 'log' )

Notice the parallel -6dB/oct slopes (characteristic of single pole filters) and the falling cut-off frequency with lengthening time constants.
The correct choice of time constant depends on how fast the variation of the desired average is. In order to capture a quickly varying mean, only signal content above the fastest frequency of variation should be attenuated -- the time constant should be chosen accordingly. This means that if the signal mean varies very slowly, long time constants are needed, and vice versa. I find that trial and error works best to determine the optimal value, but formulas to calculate the cut-off frequency do exist, e.g. see this link.
Good luck!