There are two separate issues behind your question: the duration of the measurement and the algorithm used to do the actual metering. In the audio jargon, the term used for the perceived level measurement is loudness metering.
The first issue you need to tackle is the length of the window used to do the metering. Usually, in audio they can be divided in three classes:
- Momentary, which gives a loudness level that varies quickly over time (usually a few hundreds of milliseconds)
- Short term, which winows of a few seconds
- Integrated, which gives a loudness level on the whole audio file from beginning to end
The actual values for the lengths of the windows varies from the specification, but this EBU tech note specifies a duration of 400 ms for momentary, 3 s for short-term and the whole duration of the file for integrated, of course. Moreover, if you do live tracking of the metering, there is the issue of the refresh rate of the measurement as well, but you may want to set this aside in your case.
Finally, it may be useful to note that a rectangular window is used, contrarily to other cases in audio (short-term fourier transform for instance).
To accurately represent the perceived level, you will need something more complex than a simple RMS measurement. Various methods have been developed over time, from the simple A-weighting mentioned in the comments (which is just filtering before computing the RMS value), to the LUFS (Loudness Units relative to Full Scale) measurement specified in the ITU-R BS.1770-4 recommendation. The latter is a complex algorithm, but it provides methods for measuring perceived loudness in a multi-channel audio file, which is your case.
Regarding your last questions:
- Translating the measurement to dB is a good idea to produce a more understandable chart, and most audio products that implement loudness metering give the value in dB.
- The windowing method already has inherent smoothing, because giving one value for a window of, say, 400ms time is already a kind of low-pass filtering. Regarding noise removal, most modern algorithms implement gating, i.e. an algorithm that removes all noise below a given level, which means that long periods of silence in the audio file will not lower the value measured when doing integrated measurements. It is then possible to compare the loudness from different sources (between a 2-hours movie and a 3-minutes rock song for instance).
Finally, please note (as it has been highlighted in the comments) that the value will be given in dB relative to full-scale, so it will not reflect the level of the actual audio source, except if you know the calibration of the digital-to-analog converter used for the initial recording.
There are a lot of audio plugins that implement metering, and one of the most recent ones is the Fabfilter Pro-L. You may not need such complexity for your measurement, but it can bee a good idea to take a look at what is done in the industry, the way the value is given and what a good loudness curve looks like.
You can find more comprehensive and extensive sources on loudness metering on the web, and especially on web sites of pro audio manufacturers, for instance TC Electronic or Waves