Please help me sort this issue out.

Consider a system whose output $y(t)$ is the first derivative of the input signal $x(t)$.

We can write the first derivative of an input signal as follows:

$$y(t)=\frac{dx(t)}{dt}=\lim_{\Delta t\to 0}\frac{x(t+\Delta t)-x(t)}{\Delta t}$$

Thus, we can see that the system's output must remember an infinitesimal change in future input in order to calculate the derivative, thus system possesses memory. Perfectly fine till here.

The problem comes in checking if it's causal. By the definition of the derivative above, the system has to subtract an infinitesimal future value with the current value, to calculate the derivative. Thus by definition, this has to be a non-causal system, right?

However, most of the textbooks and websites mention the first derivative as a causal system.

Please let me know if I am making any mistakes. Thank you.


Also consider the somewhat simpler "identity system", given a continuous signal $x(t)$:

$$\begin{align}y(t) &= x(t)\tag{1}\\ &= \lim_{\Delta t\to0^-}x(t + \Delta t)\tag{2}\\ &= \lim_{\Delta t\to0^+}x(t + \Delta t)\tag{3}\\ &= \lim_{\Delta t\to0}\frac{x(t - \Delta t) + x(t+\Delta t)}{2}.\tag{4}\end{align}$$

This might be interpreted as the identity system being 1) causal and anti-causal, 2) causal but not anti-causal, 3) anti-causal but not causal, and 4) neither causal or anti-causal. That's quite a set of contradictions so it must be a wrong interpretation.

Similarly, to say that the derivative of $x(t)$ exists at all points means for the derivation system that:

$$\begin{align}y(t) &= x'(t)\tag{5}\\ &= \lim_{\Delta t\to0^-}\frac{x(t) - x(t + \Delta t)}{\Delta t}\tag{6}\\ &= \lim_{\Delta t\to0^+}\frac{x(t + \Delta t) - x(t)}{\Delta t}\tag{7}\\ &= \lim_{\Delta t\to0}\frac{x(t + \Delta t) - x(t - \Delta t)}{2\,\Delta t}\tag{8}\\ &= \ldots\end{align}$$

and many other limit expressions.

The impulse response of derivation as a linear time-invariant (LTI) system is not a real-valued function of a real variable, so it is difficult to assess causality based on the impulse response. I will argue that derivation is both causal and anti-causal if the derivative of $x(t)$ exists everywhere. An applicable alternative definition of causality of an LTI system is that if input:

$$x(t) = 0\quad\text{for all }t \le t_0,\tag{9}$$

then output:

$$y(t) = 0\quad\text{for all }t \le t_0.\tag{10}$$

The derivative $y(t)=x'(t)$ is zero-valued for the constant-valued portion $t < t_0$ of $x(t)$, and because the derivative exists at $t = t_0$, then $x'(t_0) = 0$ by Eq. 6 and the natural extension of $x(t)$ in Eq. 9 to negative $t$ infinitesimally close to zero. An equivalent proof can be given for anti-causality.

This definition of causality is not useful for band-limited signals, because they will collapse to zero everywhere if they are zero-valued over an interval of any non-zero length.

Juan Antonio Fernández Madrigal also tries to answer your question in his blog, with the post title "Differentiation (derivative) is causal, but not exactly realizable". Quote:

Summary: differentiation is causal for physical signals; differentiation does not predict (actually) the future; differentiation is not (exactly and in all circumstances) realizable; differentiation can be implemented for given, carefully guaranteed cases, and only approximately if written in computer code.

| improve this answer | |
  • 1
    $\begingroup$ Good point to demonstrate that one shall not look what's inside the limit operator, but its result only. So the identity is identity. $\endgroup$ – Fat32 May 26 '19 at 18:39
  • $\begingroup$ @Fat32 or perhaps those are different systems that have equal output $y$ due to constraints on continuity of $x$. $\endgroup$ – Olli Niemitalo May 27 '19 at 6:35
  • $\begingroup$ Right. I get why the limit expressions can lead to conflicts. However, without the limit expressions, how does one go on proving that the differentiator has memory mathematically? $\endgroup$ – Nishanth Rao May 29 '19 at 2:44

If the derivative exists at the given point, then it doesn't matter if you look (infinitesimally) into the future or into the past, you can do both, because both will give the same result:

$$x'(t)=\lim_{h\to 0}\frac{x(t+h)-x(t)}{h}=\lim_{h\to 0}\frac{x(t)-x(t-h)}{h}\tag{1}$$

So a differentiator can be (theoretically) implemented by a causal system.

Clearly, the system is not stable, because a bounded input can cause an unbounded output. E.g., a step at the input causes an unbounded output signal. So if by realizable we understand causal and stable, the ideal differentiator is not realizable. However, good approximations in a certain frequency range can be realized with small errors, in the continuous domain as well as in the discrete domain.

| improve this answer | |
  • $\begingroup$ I am sorry for the confusion, but I have edited the post. The mistake was mine, I shouldn't have included the limit definition. The differentiator can be (theoretically) implemented by a non causal system also right?( Could you please read the edited post? Thank you) $\endgroup$ – Nishanth Rao May 26 '19 at 14:55
  • $\begingroup$ @NishanthARao: I've edited my answer accordingly. $\endgroup$ – Matt L. May 26 '19 at 17:38
  • $\begingroup$ Thank you so much sir. $\endgroup$ – Nishanth Rao May 27 '19 at 3:14

Just because there is an equivalence between left and right hand limit, doesn’t mean you must look both into the future and past to determine the derivative. You need one of the two to do so. Thus, the system is both causal and anti-causal. Which of course means that the system has no memory... “Infinitesimal” means so small you cannot measure it.

| improve this answer | |
  • $\begingroup$ So the system is both causal and anti-causal? However many texts say that the derivative operator has memory. This makes sense as the system must subtract a value that differs from the current value by its precision (as it is the smallest time interval it can measure, though theoretically the time interval tends to zero) to calculate the derivative. $\endgroup$ – Nishanth Rao May 26 '19 at 14:50
  • 2
    $\begingroup$ @NishanthARao: The derivative is the instantaneous change in value of the function. A theoretical derivative doesn’t need memory. In theory. In practice, yes, the time step is not infinitesimal, the system needs memory. In this case it is either causal or anti-causal. $\endgroup$ – Cris Luengo May 26 '19 at 14:53
  • $\begingroup$ Oh. Thank you so much for the response. $\endgroup$ – Nishanth Rao May 26 '19 at 14:59

[Could you provide us with more references to the assertion "most of the textbooks and websites mention the first derivative as a causal system"]

Consider a system whose output $y(t)$ is the first derivative of the input signal $x(t)$.

[Continuous] In maths, there are many possible definitions for derivatives (Gâteaux, subgradient, etc.). And when chosen, not all signals are differentiable. The assertion on the possible existence of such a system is rather strong to me. Almost a though experience. And philosophically, I believe that such a system does not exist, physically (in the most general context). So (assuming that the non-existence is granted), logically, false implies true, so the non-existing system is whatever you want: causal and non-causal, etc.

Should such a system check beforehand that any signal can be differentiated? Then, it "cannot be" not causal: it should check derivability everywhere. It cannot check causality whether a signal is right-differentiable. Even with a symbolic form: a symbolic equation should have its terms checked to see if difference rules apply.

Now, suppose you can filter inputs to continuously differentiable $\mathcal{C}^1$ signals only. Then a perfect derivative operation would be "instantaneous" (hence both causal and anti-causal).

[Discrete] But all that requires perfect continuous operations. If we now turn to approximations, for regularly sampled signals. All sampled signals can be derivable. Different discrete derivation schemes exist. Derivatives require at least two samples. With backward derivatives (like $\frac{x_{n}-x_{n-1}}{1}$), you have a causal system. With backward derivatives (like $\frac{x_{n+1}-x_{n}}{1}$), no. With two-sided ones (like $\frac{x_{n+1}-x_{n-1}}{2}$, the average of the two previous ones), neither.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.