Both answers you got regarding the cross-correlation function are quite valid and, as Hilmar states, there may be some (small or large depends on the recording system and other possibly topological and environmental factors) drifts/inconsistencies in the speeds and/or captured samples.
There are a plethora of scientific articles regarding cross-correlation and its use in the estimation of the delays, many of which are used for Direction-of-Arrival (DoA) estimation in microphone arrays.
In general, you could use the generalised cross-correlation function (for more info you can have a look at "Microphone Array Signal Processing" by Benesty, Chen and Huang, or a general Digital Signal Processing textbook such as "Digital Signal Processing - Principles, Algorithms and Applications" by Proakis and Manolakis) to estimate the delay. You could possibly do that for the whole recording if you are confident enough about the stability of your equipment, or you could possibly do the same thing for chunks of the recording in order to estimate the delay for every frame. This could provide some insight into the variability of the speed/delay between the two recordings. Of course, if you choose too small frames of data you could possibly run into erroneous estimation, as the correlation algorithms need enough data to give "good" results (they are originally derived on a statistical base).
The index of the maximum of the cross-correlation function should provide the estimate for the delay (in samples) between the two recordings, with negative values stating that the reference lags the second recording. In addition to that, many authors (for example in "Statistical and Adaptive Signal Processing - Spectral Estimation, Signal Modelling, Adaptive Filtering and Array Processing" by Manolakis, Ingle and Kogon) state that only part of the cross-correlation function is valid, as for large lags the correlation between samples is very weak for audio signals (briefly stated). Of course, since we are talking about the exact same recording this assumption may not hold for your case, but it is a good practice to "not trust" large lag values.
Furthermore, you could also have a look at a paper ("Synchronization Ambiguity in Audio Content Generated by Users Attending the Same Public Event" by Stefanakis and Mouchtaris) stating some ambiguities when trying to synchronise different recordings of the same event. This may or may not be relevant to your application though.