# Is the system $y(t) = \frac{dx(t)}{dt}$ memoryless?

The system $$y(t) = \frac{dx(t)}{dt}$$ has been shown to be not memoryless, in many texts, based on the argument that it depends on the future value of the input i.e.,$$x(t+\delta)$$ or the past value $$x(t-\delta)$$ from the definition below $$\frac{dx(t)}{dt} = \lim_{\delta\rightarrow 0}\frac{x(t+\delta) - x(t)}{\delta} = \lim_{\delta\rightarrow 0}\frac{x(t)-x(t-\delta) }{\delta}$$ My thinking is that, ideally, from the same above definition, the value of the derivative i.e., $$\frac{dx(t)}{dt}$$, is that value when $$\delta = 0$$ on the RHS(after all, that is the definition of the limit).

By this I mean to say, not at $$\delta = 0.0001$$ or at $$\delta = 0.0..(a{\space}thousand{\space}zeros)..01$$ but only at $$\delta = 0$$ (as in $$\delta$$ indeed reaches 0) the value of the expression $$\lim_{\delta\rightarrow 0}\frac{x(t+\delta) - x(t)}{\delta}$$ or $$\lim_{\delta\rightarrow 0}\frac{x(t)-x(t-\delta) }{\delta}$$ is equal to the expression $$\frac{dx(t)}{dt}$$. For any other value of $${\delta}$$, be it however small, those expressions will only be approximations as in $$\frac{dx(t)}{dt}{\approx}\frac{{\Delta}x(t)}{{\Delta}t} = \frac{x(t+{\Delta}t) - x(t)}{\Delta t}$$ where $${\Delta t}$$ is some small non-zero finite value,

and not theoritically equal to $$\frac{dx(t)}{dt}$$

So by this, the derivative of a function $$x(t)$$ at a point $$t_0$$ only depends on the value of the function at $$t_0$$ i.e., $$x(t_0)$$ and nowhere else, so it should be memoryless and in the same spirit can be said casual too(as it only depends on the present values and neither on future nor past values).

An additional perspective to support my argument above is, if we can look at $$\frac{d}{dt}$$ as an operator that takes in a function $$f(t)$$ belonging to class $$C^1$$and spits out another function $$g(t)$$ such that for every $$g(t_0)$$ there will be a corresponding $$f(t_0)$$ at $$t=t_0$$ only and no other value of $$t$$.

I would appreciate some help, please let me know if there are any mistakes in my argument. Thankyou.

• In computers we do numerical approximations. Which means we need to use the definition of the derivatives before the limit is applied. We might sample at higher rate to ensure delta t is small. This is good enough approximation that works in many engineering solutions. Commented Jul 21 at 11:04

A system is memoryless if for any $$t=t_0$$, the output $$y(t)$$ at $$t=t_0$$ can be computed from the value of the input $$x(t)$$ at $$t=t_0$$.

Example:

The system described by

$$y(t)=x^2(t)$$

is memoryless because for any $$t_0$$, the output value $$y(t_0)$$ only depends on the input value $$x(t_0)$$.

For the ideal differentiator this is clearly not the case. We cannot compute $$y(t_0)$$ from $$x(t_0)$$. In order to be able to take the limit, we need to know $$x(t)$$ in a neighborhood around $$t_0$$. It's irrelevant that $$\Delta t\to 0$$ because we can't take the limit by just knowing the value $$x(t_0)$$. In other words, there's a difference between infinitesimal and zero. So you could say that an ideal differentiator has an infinitesimal memory but it is not memoryless. However, it can be causal, depending on the definition of the difference quotient.

I think your misunderstanding comes from ignoring the difference between limits and function values. You claim that only for $$\Delta t=0$$ does the difference quotient

$$\frac{x(t+\Delta t)-x(t)}{\Delta t}\tag{1}$$

equal the derivative $$dx/dt$$. However, that is false because $$(1)$$ doesn't exist for $$\Delta t=0$$. Its limit for $$\Delta t\to 0$$ may exist, and if it does, it equals $$dx/dt$$.

[...] the derivative of a function $$x(t)$$ at a point $$t_0$$ only depends on the value of the function at $$t_0$$, i.e., $$x(t_0)$$ [...]

is wrong and I think it is the root of your misunderstanding. The value of $$x(t)$$ at $$t=t_0$$ says nothing about the derivative of $$x(t)$$ at $$t_0$$. It is straightforward to find infinitely many functions with the same value at $$t_0$$, all with different derivatives at $$t_0$$. E.g., all functions

$$x(t)=a(t-t_0)+b$$

with fixed $$b$$ have the same value $$x(t_0)=b$$, but their derivatives at $$t=t_0$$ (and everywhere else) are equal to $$a$$, which can be chosen arbitrarily.

• But doesn't we talk about derivative of a function at a point as the "slope of the function at that particular point". Sure we can't calculate the slope at a single point, but by the definition of a derivative, although we take an infinitesimally colse-by point to the original to find the slope, it then is pulled back closer to the actual point to make better and better approximations and so whatever the slope we get at the end is, by definition of d/dt, at the original point alone. Am I missing anything here.
– Guna
Commented Jul 21 at 10:47
• "Sure we can't calculate the slope at a single point", correct, case closed, @10GunaSekhar! That's what matters. Everything else does not matter. Commented Jul 21 at 12:26
• @10GunaSekhar: just look at $y = x$ and $y = x^2$ Both have $y(1) = 1$. If differentiation would be memoryless they would have to have the same derivative at $x=1$ which they clearly don't. Commented Jul 21 at 12:45

In taking the limit, you are going from discrete to continuous time. Taking the derivative of a function in continuous time is not as innocuous as you think. The derivative inherently relies on past values of the function.

First let's define a memoryless system as follows. A memoryless system can be seen as something like:

$$y(t) = f\big(x(t)\big)$$

However let's consider:

$$y(t) = \dfrac{d}{dt}x(t) = x'(t)$$

However, in order to prove that this system is memoryless, we would need to put $$y(t)$$ as a function of only $$x(t)$$, not $$x'(t)$$ from our definition. Is there a way to express $$x'(t)$$ in terms of $$x(t)$$ then? In fact there is using the Fourier Transform.

If we take the Continuous-Time Fourier Transform:

$$\mathscr{F}\left\{\dfrac{d}{dt}x(t)\right\} = \underbrace{j\omega}_{H(\omega)} X(\omega)$$.

Ah! So we can think of differentiation as being some sort of system being applied to $$x(t)$$. The ideal differentiator then is $$H(\omega) = j\omega$$. We know that multiplication in the frequency domain is convolution in the time domain, but if it's a convolution, the behavior at any given point is given by an integral between the impulse response of our differentiator and the signal itself, which itself isn't a function of $$x(t)$$ alone anymore. But just like an ideal lowpass filter, we cannot implement it in this form.

We can approach this ideal filter using a variety of schemes by sampling more and more points of $$x(t)$$, but all of those will involve convolution of multiple points, so the differentiator in reality isn't a function of just a single point in $$x(t)$$ at all! Our notation just obscured that from us. It makes sense doesn't it? The behavior of the derivative inherently depends on the points around it. The derivative is about measuring change, and you can't assess change without remembering what the function has been doing.

What are these schemes? They're in general finite difference methods. As we increase the number of coefficients in our FIR filter, we approach the true derivative.

My infinitesimal 2 cents:

$$y(t) = \frac{\mathrm{d}x(t)}{\mathrm{d}t}$$

is the same as the system

$$x(t) = \int\limits_{-\infty}^{t} y(u) \, \mathrm{d}u$$

except reversing the roles of $$y(t)$$ and $$x(t)$$.

This could be done in continuous time with a capacitor having unit capacitance (in whatever measuring system you're defining $$x(\cdot)$$, $$y(\cdot)$$, and $$t$$ with) and $$y(t)$$ is the current going into the capacitor and $$x(t)$$ is the resulting voltage.

This capacitor is integrating charge flowing into it and that remembers the previous charge and adds the new charge $$y(t)\mathrm{d}t$$, every infinitesimal time period of width $$\mathrm{d}t$$.

Even in the infinitesimal world where $$\mathrm{d}t \to 0$$, that requires memory.