How to remove ADC-induced noise from voltage measurement data?

Question

I am measuring the various voltages involved in a solar energy system, but the recorded values vary by quite a bit, most of which is caused by the ADC in the Arduino Nano I am using to take the measurements.

I am taking readings of both 30 Volt and 100 Volt sources (batteries and solar panels) on separate ADC channels, using hardware voltage dividers to present the readings to the ADC as a voltage from 0 to 5 volts.

The ADC is specified as having 10-bit resolution and initially presents the values as an integer from 0 to 1023. The readings are communicated as floating-point values after a conversion has been applied to scale the recorded readings up to the original voltages.

What is the best way to smooth these out so I can do a data reduction?

My raw data files include date and time, (to the millisecond level), with readings taken every 2000 ms.

The files become quite large so I would like to eliminate redundant readings if possible.

The presence of date/time information will allow for continuous graphing.

Ideas?

Answer 1

So, first of all, Nyquist.

I.e. when doing real-valued sampling, you must be 100% certain the analog signal you observe has less than half your sampling rate in bandwidth, or else you get all the signals from outside that bandwidth aliased into your observed band.

In your case: if your power might change more than once every 4s (which it very likely can do), you get unusable measurements. Add an analog low-pass filter in front of the ADC that cures that problem. This step is not optional; when you sample something every 2s, ie. with a sampling rate of 0.5 Hz, you need to filter it down to 0.25 Hz.

Because building a 0.25 Hz filter is a boring thing (for example, if you do that as a simple RC filter, your R and your C need to be pretty large - large capacitors are bulky, leaky and inaccurate in value, and large Resistors mean that you probably need to buffer your signal to be able to reliably drive a current through that resistor first), I'd recommend increasing your sampling rate, so that filtering becomes easier; a 500Hz filter for an Arduino that takes one sample every 1ms is easier to build, and then you'd do the filtering down to the bandwidth you actually need in software on the Arduino – and that can be done in a number of ways, for example using the moving average you've mentioned, or using Biquad IIRs (to save on filter coefficient RAM) or simply using decimating FIR filters (which can be made to run at the output, not the input sampling rate).

Your Date/Time information is superfluous. Safe the time of start. After that, you define the sampling rate by initiating sampling on a timer interrupt (or by having an ADC module that does that automatically – I don't know the Arduino well enough to know what works there). So no need to store the time – simply the sample count divided by the sample rate gives you that time.

If you store to a fast, large medium like an SD card and don't have to worry about power consumption of writing to that: Why do any signal processing on the Arduino itself (again, you must first filter your analog signal to adhere to Nyquist)? 1 kSa/s of let's say 16 bit values is 7200 B/hr = 169 kB/day . Assuming you have a 2GB card... that's a lot of days.

If you instead plan to send the data directly to something like a PC, do the processing there, not on the Arduino.