A single moving average filter such as a first order CIC with decimate by D (which is mathematically equivalent to a moving average over D samples and selecting every Dth sample) has a frequency roll-off for large D that approaches 6dB/octave, consistent with a first order filter.
The frequency response for a moving average is an aliased Sinc function (approaches a Sinc as D approaches infinity), and the peaks of the Sinc roll-off at 1/f which is the -6 dB/octave roll-off previously mentioned.
The OP mentioned that the manufacturer recommended a 4th order filter. This is likely due to the order of the Delta Sigma modulator such that if a lower order low pass filter was used, it would be insufficient to filter out the increasing noise floor due to noise shaping.
A possible solution, assuming the decimation rate is high enough (meaning the resulting bandwidth of the desired signal can be low enough) is to use a 4th order CIC which is implemented by cascading four accumulators, then selecting every Dth sample, and cascading four "combs" or differences:
These are implemented in fixed point with wrap on overflow and with sufficient precision that any accumulator only overflows once in the D sample duration. This is efficiently mathematically equivalent to cascading four moving averages over D samples, and then after the averaging, selecting every Dth sample as depicted below (and it could also be implemented this way; as noted the CIC does the same thing and more efficiently).
(To mention, the response for the above filter approaches that of a single FIR filter with Gaussian weighted coefficients; higher order moving average filters and higher order CIC filters are essentially Gaussian filters!). Shown below are example magnitude frequency responses of this with using a four stage moving average with $D=200$ and $D=50$ samples (and prior to the final decimation). Note how the first null is at a normalized frequency of $2\pi D$ radians/sample.
The red line shows the 4th order -80dB/decade roll-off that is approaches for large $D$, and the effect of aliasing at Nyquist. The appropriate duration for averaging to use depends on the highest bandwidth of interest for the observed signal (how fast is the acceleration expected to be changing?) and the stability of the accelerometer: averaging duration should not exceed the Bias Instability floor for the Allan Deviation plot of the accelerometer (at which point the result of averaging would be worst than with a shorter duration!). See Freescale's App-not AN5087 for related info.
Further details:
A traditional data converter has a quantization noise spectrum that is well modelled and approximated as a uniform white noise process (uniform in magnitude distribution in contrast to Gaussian, and white as also uniform across the frequency spectrum), which means the power spectral density due to quantization noise is flat over the unique range of the digital spectrum (Nyquist or from DC to half the sampling rate for the spectrum of a real signal). Graphically the spectrum for a quantized sinusoidal waveform would appear as follows:
The peak shown represents a single sinusoidal tone at full scale, and the noise floor is based on the total number of bits used in the conversion. The relationship between the total noise and signal power of the sinusoid is given by the formula as derived and further detailed at this link:
$$SNR = 6.02 \text{ dB/bit} + 1.76 \text{ dB}$$
A good example of a comparative spectrum for a Delta Sigma converter is provided by this paper: "Single low-gain amplifier compensated hybrid delta-sigma modulator" by Arshad Hussain and Goang Seong. Here they implemented a 3rd order delta-sigma, and the Noise Transfer Function (NTF) which is the shaping of the quantization noise compared to a traditional converter, as well as the resulting power spectral density, is copied below:
Note that consistent with a 3rd order noise shaper, the power spectral density is increasing 6x3=18 dB/octave or 60 dB/decade. As detailed in the link provided earlier, a traditional converter provides an oversampling improvement in quantization noise of 10 dB/decade (or 3dB/octave which is 1/2 a bit for every doubling in frequency), so this third order, if properly filtered, would provide an oversampling improvement of 70 dB/decade (or 21 dB/octave or 3.5 bits for every doubling in frequency!). Proper filtering is the key word. In the example given with a tone at a normalized frequency of approximately $0.01\pi$ radians/sample, we would want to use a low pass filter to remove the noise that has been pushed to the higher frequencies, with a cutoff above that, and then that filter should have a roll-off greater than -60 dB/decade. If the filter roll-off was exactly -60 dB/decade (consistent with a 3rd order filter), the remaining noise spectrum above $0.01\pi$ radians/sample would be flat such that continued additional filtering could provide the same improvement we get in a traditional converter of 3dB / octave (1/2 a bit for every doubling). If the filter roll-off was -80 dB/decade (consistent with a 4th order filter), there would be little benefit of additional filtering given the resulting noise itself would be going down -20 dB/decade already.
