I think it all depends on the SNR region that you are dealing with. Using a voice activity detector (VAD) seems to be worth an attempt, but consider that VADs fail if the SNR is too low. The most simple approach might be using quantiles of the power distribution. Some lower quantiles will probably not contain power that is caused by the speech signal part while high quantiles will contain power resulting from the speech signal. Choosing the right quantiles might be a challenge, but due to the straight forward approach it might by worth a try...
Furthermore, by reinterpreting your problem to noise floor estimation, you might succeed by using some noise floor estimation algorithm (e.g. Minimum-Statistics by Rainer Martin [1, 2]) and calculating the SNR from that.
 Rainer Martin, “Spectral Subtraction Based on Minimum Statistics,” presented at the EUSIPCO, 1994.
 Rainer Martin, “Noise Power Spectral Density Estimation Based on Optimal Smoothing and Minimum Statistics,” IEEE Transactions on Speech and Audio Processing, vol. 9, no. 5, pp. 504–512, Jul. 2001.