2
$\begingroup$

I showed the question about flipping the impulse response to some math teachers and electrical engineers at an university. They read Dilip's answer and thought that it was hard to understand what Dilip meant and it was simply not true...

Could someone confirm that it really isn't true what Dilip Sarwate wrote. And explain it in a way that is easier to read.

See: Flipping the impulse response in convolution

There is no "flipping" of the impulse response by a linear (time-invariant) system. The output of a linear time-invariant system is the sum of scaled and time-delayed versions of the impulse response, not the "flipped" impulse response.

$\endgroup$
  • 2
    $\begingroup$ Dilip's answer is correct. I thought it was a rather good explanation - going back to the basics of a LTI system definition... A better way to explain it would maybe be through graphics/animations, but it would be a lot of work to prepare this just to clarify an answer! $\endgroup$ – pichenettes Apr 6 '13 at 13:41
  • 1
    $\begingroup$ I agree with pichenettes. I have voted to close this, as it's not a real question. $\endgroup$ – Jason R Apr 6 '13 at 15:41
  • 1
    $\begingroup$ @DilipSarwate: "Go ask your teachers to explain any animations that you have found" - What do they SUPPOSE TO tell me when I ask that? (Or about simulations like this one: jhu.edu/signals/convolve ) Please elaborate on that. Please confirm whether - in your own opinion - there's something wrong with the videos or not. $\endgroup$ – user68610 Apr 6 '13 at 23:22
  • 2
    $\begingroup$ There are two ways of explaining convolution: one is to explain what's really going on (decomposing the input into impulses and summing the scaled/time-shifted impulse response using the LTI property), just as Dilip or BZ did. The other is to work your way to the convolution formula (which includes a minus sign) and try to illustrate this formula a posteriori. The videos and applet you have linked to are in this second category. They appear to be technically correct, but I don't think they give a lot of insight about what's happening. $\endgroup$ – pichenettes Apr 7 '13 at 0:32
  • 2
    $\begingroup$ Cross-post: scicomp.stackexchange.com/questions/6774/…. I am a mod for SciComp; the question there was closed in favor of the question here. $\endgroup$ – Geoff Oxberry Apr 7 '13 at 1:20
6
$\begingroup$

Flipping the impulse response is really just a matter of perspective. The LTI system doesn't care about perspective. In any case, here is a graphic showing a system that takes an input of color weighted impulses.

enter image description here

$\endgroup$
  • 3
    $\begingroup$ @DilipSarwate - That commutative law is just too confusing for me. $\endgroup$ – user2718 Apr 7 '13 at 19:27
  • 3
    $\begingroup$ @DilipSarwate - But don't the colors really pop? $\endgroup$ – user2718 Apr 7 '13 at 19:29
  • $\begingroup$ @BZ : Post this on the other question, too, and I'll award the bounty! :-) $\endgroup$ – Peter K. Apr 7 '13 at 21:00
  • 1
    $\begingroup$ @PeterK. Thanks, but no need. I think the discussion has served its purpose. $\endgroup$ – user2718 Apr 7 '13 at 21:53
1
$\begingroup$

A convolution is a computational trick where a "flipped" impulse response is used to compute how all the actual unflipped impulse responses due to input back in historical time would have summed up to create a current response, had they not already been doing so by an LTI system carrying its own state. The same computational trick could be performed by flipping the input (by index) in time, instead of the impulse response, but people don't like doing this as much. This computational trick assists in the use vector processors that only step down multiple vectors via a single incrementing index.

An LTI state machine, left on its own, and running forward in time, executes down the path of un-flipped impulse responses already initiated (scaled) by actual previous input to the system.

The LTI system does no flipping. But if you want to see what it would have done on some input that isn't currently represented by its actual current state, you might use this computational "flipping" trick on either the impulse response or on the time domain input.

$\endgroup$
1
$\begingroup$

You're on a mountain top, surrounded by a few other peaks, and shout "HEY!!". 2 seconds later you hear "HEY" (10% softer), and another second later an even softer "hey" (1% softer). So the impulse response of the system is a dirac at t = 2 with a weight of 0.1, plus another dirac at t = 3 with a weight of 0.01.

Now you say "ECHO!!", at $t = 0$ and the whole utterance takes a bit more than 1 second. How to determine, at a given point $t$ in time, say $t = 3$, what will be heard from the echo? (Answer: a mixture of a loud "O" and a softer "E").

One way is to figure out if something has been shouted 2s ago (in which case you hear it attenuated by 0.1), and if something has been shouted 3s ago (in which case you hear it attenuated by 0.01). This could be written as:

$y(t) = \int h(\tau)x(t-\tau)d\tau$

"For each tap of the impulse response at $\tau$, I'm looking in the past of the input $x(t-\tau)$". This is the "scaled and time-shifted impulse response" view. Dilip's post, BZ's post.

Another way, is to say, "OK, let's play back a recording of what has been shouted originally. I hear "E" at $\tau = 0$. I'm interested in knowing what's happening at $t = 3$. I will check how strong the echo is after $3$ seconds. Then I hear "O" at $\tau = 1$. How strong is the echo after $2$ seconds? This could be written as:

$y(t) = \int x(\tau)h(t-\tau)d\tau$

"I am retracing the history of the input $x(\tau)$ and read from the impulse response by how much each sample is weighted to affect the present - the quantity $h(t - \tau)$".

Both are equivalent because they are just the same integral with a variable substitution - convolution is commutative. None of them imply that mountain echoes have time-traveling properties. None of them imply that you'll hear a softer "hey" before you hear the louder "HEY". None of them imply that you'll hear your voice backwards.

What changes is just the order in which you access the history of the input to compute the output - in reverse-chronological order ; or in chronological order.

A way of writing things in a way that does not show a minus sign:

$y(t) = \iint\limits_{u, v, u+v = t} x(u) h(v) du dv$

Let me beat this dead horse one last time: when you do a long multiplication (this is indeed a convolution of the sequence of digits, with some carrying over), you will process digits in a different order whether you write it as $A \times B$ or $B \times A$. The result is the same and no number gets "written backwards" in the process.

$\endgroup$
  • $\begingroup$ I like the double integral thing. Is that in use in some particular theoretical framework? $\endgroup$ – user2718 Apr 8 '13 at 13:04
  • $\begingroup$ It reminds me of wave propagation problems (room acoustics, models for cellular network planning in urban environments...) in which you sometimes write integrals over all paths that take a given amount of time to be traversed. $\endgroup$ – pichenettes Apr 8 '13 at 13:27
1
$\begingroup$

I'll try my hand at this also. In my experience, in courses where the concept of convolution is taught, it's common to see the "flip-and-slide" process diagrammed out as you've alluded to. I don't feel that the graphical illustration itself really gives that much insight, but here's how you might arrive at it starting with the convolution integral itself:

$$ y(t) = \int_{-\infty}^{\infty} x(\tau)h(t-\tau)d\tau $$

In the event that the impulse response $h(t)$ is causal (i.e. it is zero $\forall t < 0$, and it usually is), then the above integral can instead be written as:

$$ y(t) = \int_{-\infty}^{t} x(\tau)h(t-\tau)d\tau $$

Note that you can change the limits of the integral, because $\forall \tau > t$, the argument to $h(t)$ is less than zero, therefore making the integrand zero. Likewise, if the input signal $x(t)$ is causal, then the above integrand is zero $\forall \tau < 0$, simplifying it further to:

$$ y(t) = \int_{0}^{t} x(\tau)h(t-\tau)d\tau $$

This is starting to look a bit less imposing. Now, the question is, how does this equation lead to the "flip and slide" interpretation of convolution? Well, by straightforwardly reading the above equation, you can recognize the following:

  • In order to calculate the output of the convolution $y$ at some point in time $t$, we sum up a bunch of terms over the variable of integration $\tau$, ranging from $0$ to $t$.

  • Each term consists of the product of the impulse response evaluated at the variable of integration $\tau$ and the input signal, time-reversed with respect to the variable of integration $\tau$ and shifted to the left by the current output time $t$. This is where the concept of "flipping" the impulse response as an interpretation of the convolution integral comes from.

However, understand one thing: there is no actual time reversal of the impulse response that occurs when you excite an LTI system with an input signal. That's merely one way of intuitively interpreting the structure of the convolution equation, and one that lends itself well to a graphical explanation that most will probably have seen in an undergraduate signals and systems course. One such example is given on Wikipedia, duplicated below. It is interpreted as follows:

$\hskip1in$enter image description here

  • When you see the impulse response "sliding" across the input signal, that illustrates the progression of the time variable $t$.

  • At any particular time instant instant $t$, in the animation, remember that the output signal is equal to $$ y(t) = \int_{-\infty}^{\infty} x(\tau)h(t-\tau)d\tau $$ I kept the generic form here to better fit the diagram on Wikipedia, which is not causal. We take the product of the input signal $x(\tau)$ and the time-reversed, shifted impulse response $h(t-\tau)$ and integrate over the range of $\tau$ where the two functions overlap.

  • This action is equivalent to finding the area under the curve formed by the product $x(\tau)h(t-\tau)$. This area is shown as the yellow area in the animation.

  • At any time $t$, the output of the system $y(t)$ is equal to the amount of area shown in the animation at that particular time.

As I said, that's the description that you've probably gotten from your professors, and it works well if you're tasked with calculating convolution integrals for simple functions, which typically involves some variation of the above process over a few piecewise intervals. Like I said before, though, I don't feel like this interpretation gives you much intuition to go on.

In my opinion, the superior explanation is that espoused by Dilip in the linked answer. You can arrive at it via what I think is an even more straightforward reading of the convolution integral:

$$ y(t) = \int_{0}^{t} x(\tau)h(t-\tau)d\tau $$

Read it as follows:

  • The time function $y(t)$ consists of the sum of a bunch (an infinite number, actually) of functions of $t$.

  • Each term in the sum has the form: $$ x(\tau)h(t-\tau) $$

  • This is a function of $t$, expressing a copy of the system's impulse response $h(t)$, delayed in time by $\tau$ (the variable of integration) and weighted by the input signal $x(t)$, evaluated for $t=\tau$ (again, the variable of integration.

  • So, intuitively, we form the output signal $y(t)$ by adding an infinite number of copies of the impulse response. Each copy of the impulse response is delayed by $\tau$ and weighted by the input signal evaluated at that value $\tau$. For the causal case, $\tau$ ranges from zero to $t$.

That's it! I feel that this gives a much better feel for the "action" that an LTI system performs to an input signal. Think of it this way (which is very mathematically imprecise): at each time instant, the input signal "causes a copy of the impulse response to start coming out of the system," with an amplitude commensurate with the value of the input at that instant. Due to the linearity and time invariance of the system, all of these impulse response copies sum up to form the composite that is the output signal $y(t)$. While I find this very intuitive, it is even more clear for the discrete-time case: $$ y[n] = \sum_{k=0}^{n} x[k] h[n-k] $$ This is the exact same concept, except the infinitesimally-spaced integrals turn into discrete sums (and the discrete nature makes it even easier to visualize, as Dilip tabulated it in his answer that you referenced). I think that using this idea is the best way to express the nature of the convolution integral.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.