Well, you'd look at the spectrum of your noise, and deduct an appropriate filter frequency response from that. When doing so you'll realize that the moving average is not a good filter (having sinc-shaped frequency response), and that you should be using a better filter design, aligned to what your signal actually needs.
So, you'd do three things:
- you come up with a physical model of how fast the acceleration changes
- you verify that with a look at the spectrum of your recorded signal, and look for where the noise in your recording is
- you design a filter that has a passband where your expected signal is, and especially strong stopband suppression where only noise happens (hint: many accelerometers are swinging micromachined things, and there might be significant noise from that at subharmonics of the swing frequency)
Also, this is the classical example for which Kalman filters (or nonlinear variants of it, depending on how you plan to model your state) are used, instead of plain double integration.