Read the original paper: Schmidt, R. O. "Multiple Emitter Location and Signal Parameter Estimation." IEEE Transactions on Antennas and Propagation. Vol. AP-34, March, 1986, pp. 276–280
You may also want to look up "Pisarenko's Method", "Prony's Method" and read about related problems such as ESPRIT (Roy, R.; Kailath, T. (1989). "Esprit - Estimation Of Signal Parameters Via Rotational Invariance Techniques". IEEE Trans. Acoustics, Speech and Signal Processing 37 (7): 984–995.)
You can find a lot of this in statistical signal processing books like "Statistical Digital Signal Processing and Modeling" by Monson Hayes and look at the cited references therein. Another reference is Mathematical Methods for Signal Processing by Moon & Stirling.