A simple approach to doing this would be to window first in the time domain (to avoid spectral leakage issues) and the root sum square the bins from the FFT that are within each frequency band of interest, while properly scaling for the window loss. This would work out to be the following equation which involves a minimum amount of processing and provides for accurate power estimates as long as the signal energy is spread over multiple bins:
$$P_b = \frac{\sum_k(X[k]X^*[k])}{N \sum\left(w[n]^2\right)}\tag{1}\label{1}$$
Where
$P_b$: the total noise power in input units squared
$k$: bin numbers within a band of interest
$N$ total number of bins in the DFT
$w[n]$: N samples from window used
Equation $\ref{1}$ assumes a DFT computed as:
$$X[k] = \sum_{k=0}^{N-1}x[n]e^{-j2\pi nk/N}$$
DETAILS
Each bin in the FFT is a bandpass filter with an equivalent noise bandwidth of 1 bin, so you could be tempted to null all bins that are not within a 1/3 Octave range of interest and then root-sum-square the bins. This would be very accurate if your noise was white but leads to spectral leakage issues when you have a large difference between weaker and stronger signals at different frequencies in your waveform. Nulling frequency bins is the equivalent of the "Frequency Sampling" approach to filter design and has the worst performance as detailed further here:
Why is it a bad idea to filter by zeroing out FFT bins?
One approach would be to design a series of bandpass filters but given you are only interested in average power of each band, a simpler approach is to first window the waveform in the time domain prior to taking the DFT and then root sum square the bins of interest in each band. This increases the bandwidth of each bin of the resulting DFT, such that it's value is now proportional to the total equivalent power of more than one bin, so this factor must also be compensated for (scaling) but has the significant benefit of rejecting signals much further away. My favorite window is the Kaiser Window since with that one you can vary the windowing parameter that trades this bandwidth (frequency resolution) with rejection.
Below demonstrates the frequency response of one bin for a rectangular window over a shorter 20 sample sequence (no additional windowing) versus the Kaiser window with windowing parameter 12. The resulting bin magnitude would be proportional to the spectral content of all areas under each of these curves, so we see how the rectangular window is much more sensitive to frequency content that is further away, but has a much tighter frequency resolution as well.
I further detail the approach on compensating for the increased bandwidth and the overall power reduction due to the window at this post here:
How to calculate resolution of DFT with Hamming/Hann window?
