First the simple questions: Is there an effect on the Nyquist frequency when I apply a moving average filter on the raw data before I downsample? And what does this do to aliased frequencies?
Background of my question, to help you answer in a way that I can understand it:
I am working with very large amounts of data from a production operation (flow, temperature, pressure, etc.). The data is collected at a ridiculously high sampling rate, and also the sampling and data are not fully reliable: sometimes values are missing, measurement artifacts cause spikes in the time trends, and often there is variation in the sampling frequency. This makes any analyses "unpleasant".
I started with downsampling by simply taking 1 min spaced values from the database, and I worked in Excel because it helped us try stuff out quickly. We gained experience with all the calculations we do, and we are now moving to different software. With this change, I also would like to take a better approach to determining a suitable sampling rate. I am completely untrained and inexperienced with DSP, so I read an introductory text. Now I know that I have to think about the Nyquist frequency. In a messy system like this, there are low frequencies that I would like to study, while there are also things happening at high frequencies (such as the aforementioned spikes) that I would rather filter out.
The new software has two options. The first is to simply return the value that happens to be in the database at the sampling rate. This is what I did to get to Excel. It has the downside that the missing values return a (NULL) result and handling these is a pain in the behind. The second is to return the average value over the length of each sampling period. I think this is equivalent to applying a moving average filter to the raw data, and then resampling at the same rate as the MA length. This method is more robust (returns (NULL) much less often) and mitigates the effect of artifacts.
I am ready to choose between the two downsampling methods, and to make a proper selection of suitable sampling rates rather than reducing the rate by trial and error until the results "don't look right". So, any insights would be appreciated.
Thank you for your time.