RMS is essentially a three step process on a signal: Square it, mean it, root it.
Squaring is not so difficult in software. Two things to keep in mind here. First, squaring fixed point data will double the data width, so make sure you have enough precision to avoid overflow. Second, squaring a signal will double its bandwidth. Make sure you have enough margin in your sample rate to either eliminate aliasing, or that the aliasing doesn't affect your baseband signal.
A square root can be computationally expensive, but it isn't exactly difficult to do. I does have an impact on bandwidth, but this could be treated as a data representation problem, for which you would not consider it to affect bandwidth. I wouldn't worry about it.
A mean technically requires the full data set, so it doesn't apply to real time processing. To get around this, we can instead apply low pass filtering. Low pass filtering is a big enough topic that I don't think it warrants much discussion here, but you could any standard filtering techniques, such as IIR, FIR. I would recommend trying some and maybe asking another question on here if you need more information.
It isn't explicitly required, but it is likely that you would need to decimate your signal at some point. For example, your sample rate may be 1MHz, but the user interface might only refresh at 100Hz. This would happen either after or during your filtering stage. You could apply a pair or second order maximally flat IIR low pass filters with a cutoff of say 25Hz, then throw away 9,999 out of every 10,000 samples. You could also save 10,000 samples in a buffer at a time, and calculate the average of this set. This is (I believe) how an oscilloscope does its math, but keep in mind that the accuracy wouldn't be great, because the filter is not aggressive enough for the decimation. You could 'smooth' the data by applying a window to the buffered data, but you would still get some aliasing if your buffers do not overlap.
Lastly, the bandwidth of the input signal. At 1MHz, 500kHz is at Nyquist, so it is the absolute maximum bandwidth you can sample without aliasing. However, this doesn't provide any margin, so your sampling is likely to be sub par because of real world sampling limitations, such as quantization error, and noise. Additionally, squaring a 500kHz would create content at 0Hz as well as 1MHz. The 1MHz content would alias back down to 0Hz, causing distortion of the 0Hz signal. As such, you would want to build in some margin to avoid aliasing.
All this being said, I don't know how much proper signal processing technique goes into your off the shelf DMM. In fact, it is possible to do a fair bit of this process in the analog domain to offload processing power. It seems more likely that someone designing a DMM would be inclined to throw high precision, high bandwidth ADCs at the problem and brute force a solution that provides a number. That number might be accurate for a steady state sine wave at a given frequency range, but inaccurate for real world signals. However, it is unlikely that the user would either notice or care.