# Continuous-time RNN and Shannon sampling theorem

The most-used discrete-time RNN equations used in Deep Learning these days are those of Elman:

I have seen two very different continuous-time version of these, with different justifications. The most "natural" to me are the leaky integrator onces (here sigmoid is replaced with tanh, but any activation function would do):

which is justified by saying that taylor expanding this equation with discretization step of 1 gives back the discrete time equations above (see this).

Another version is found on Wikipedia (notice how the weight is outside the activation function!)

and justified by saying

Note that, by the Shannon sampling theorem, discrete time recurrent neural networks can be viewed as continuous-time recurrent neural networks where the differential equations have transformed into equivalent difference equations. This transformation can be thought of as occurring after the post-synaptic node activation functions $$y_{i}(t)$$ have been low-pass filtered but prior to sampling.

Could someone explain this?