If we convolve 2 signals we get a third signal. What does this third signal represent in relation to the input signals?

  • 4
  • It's just "reverse, shift, multiply and sum", no more. – chaohuang Oct 23 '12 at 7:55
  • It has the meaning of "averaging",in signal processing terms that could mean removing high frequency components. – Giwrgos Rizeakos Feb 14 '14 at 10:22
  • 2
    I've been searching for a resource that explains the physical meaning of convolution for a while now and I finally found it here: colah.github.io/posts/2014-07-Understanding-Convolutions – faridghar May 27 '16 at 15:18
  • consider any physical or mechanical system, then the input to the system is x(n), the parameters that we aleady defined is h(n). the workfunction for the system to work is y(N)=x(N)*h(N) ex: consider sinewave as input to a multiplier generally multiplier works with so many analog circuits right, we can change the device perfomance by simply varying parameters then the parameters we pass are h(n) – user6151179 Nov 15 '16 at 15:40

16 Answers 16

up vote 35 down vote accepted

There's not particularly any "physical" meaning to the convolution operation. The main use of convolution in engineering is in describing the output of a linear, time-invariant (LTI) system. The input-output behavior of an LTI system can be characterized via its impulse response, and the output of an LTI system for any input signal $x(t)$ can be expressed as the convolution of the input signal with the system's impulse response.

Namely, if the signal $x(t)$ is applied to an LTI system with impulse response $h(t)$, then the output signal is:

$$ y(t) = x(t) * h(t) = \int_{-\infty}^{\infty}x(\tau)h(t - \tau)d\tau $$

Like I said, there's not much of a physical interpretation, but you can think of a convolution qualitatively as "smearing" the energy present in $x(t)$ out in time in some way, dependent upon the shape of the impulse response $h(t)$. At an engineering level (rigorous mathematicians wouldn't approve), you can get some insight by looking more closely at the structure of the integrand itself. You can think of the output $y(t)$ as the sum of an infinite number of copies of the impulse response, each shifted by a slightly different time delay ($\tau$) and scaled according to the value of the input signal at the value of $t$ that corresponds to the delay: $x(\tau)$.

This sort of interpretation is similar to taking discrete-time convolution (discussed in Atul Ingle's answer) to a limit of an infinitesimally-short sample period, which again isn't fully mathematically sound, but makes for a decently intuitive way to visualize the action for a continuous-time system.

  • 1
    The integrand should be $x(\tau) h(t-\tau).$ – Atul Ingle Oct 20 '12 at 20:24
  • You beat me to it. Thanks. – Jason R Oct 20 '12 at 21:00
  • 1
    But this is not just shifting the signal and then taking the output sum , one of the signal is first reversed unlike correlation.And if one of the signal is the impulse response then it is ok , but what if two signals are different like convolving 2 images. – p.j Oct 21 '12 at 15:23
  • @ParagJain: Actually, it is just taking the sum of many (actually an infinite number) of scaled copies of the impulse response. The impulse response term $h(t-\tau)$ is not reversed with respect to the time variable $t$. For cross-correlation, as you noted, the sign on $\tau$ inside the impulse response argument is changed to a $+$. It doesn't matter whether $h(t)$ is really the response of an LTI system or not; it could be an image as you proposed or any signal for that matter. The smearing action created by convolution is the same, as described in the answers here. – Jason R Oct 22 '12 at 14:17

A particularly useful intuitive explanation that works well for discrete signals is to think of convolution as a "weighted sum of echoes" or "weighted sum of memories."

For a moment, suppose the input signal to a discrete LTI system with transfer function $h(n)$ is a delta impulse $\delta(n-k)$. The convolution is \begin{eqnarray} y(n) &=& \sum_{m=-\infty}^{\infty} \delta(m-k) h(n-m) \\ &=& h(n-k). \end{eqnarray} This is just an echo (or memory) of the transfer function with delay of k units.

Now think of an arbitrary input signal $x(n)$ as a sum of weighted $\delta$ functions. Then the output is a weighted sum of delayed versions of h(n).

For example, if $x(n) = \{1, 2, 3\}$, then write $x(n) = \delta(n) + 2 \delta(n-1) + 3 \delta(n-2)$.

The system output is a sum of the echoes $h(n)$, $h(n-1)$ and $h(n-2)$ with appropriate weights 1, 2, and 3, respectively.

So $y(n) = h(n) + 2h(n-1)+3h(n-2)$.

  • 1
    It could be very helpfull to add a graphical representation of each of your equations. Some people (such as myself) have a better understanding of convolution when looking at it visually. – PhilMacKay Oct 22 '12 at 21:15
  • 1
    @PhilMacKay For a graphical representation (actually a tabular representation) of discrete convolution, see this answer. – Dilip Sarwate Nov 19 '15 at 16:23
  • 1
    "echoes" should really be replaced by "memories". Remembrance of Things Past :Proust. – rrogers Apr 6 '16 at 15:47

A good intuitive way of understanding convolution is to look at the result of convolution with a point source.

As an example, the 2D convolution of a point with the flawed optics of Hubble Space Telescope creates this image:

enter image description here

Now imagine what happens if there are two (or more) stars in a picture: you get this pattern twice (or more), centered on each star. The luminosity of the pattern is related to the luminosity of a star. (Note that a star is practically always a point source.)

These patterns are basically the multiplication of the point source with the convoluted pattern, with the result stored at the pixel such that it reproduces the pattern when the resulting picture is viewed in its entirety.

My personal way of visualizing a convolution algorithm is that of a loop on every pixel of the source image. On each pixel, you multiply by the value of the convoluted pattern, and you store the result on the pixel which relative position corresponds to the pattern. Do that on every pixels (and sum results on every pixels), and you get the result.

You can also think of convolution as smearing/smoothing of one signal by another. If you have a signal with pulses and another of, say, a single square pulse, the result will the smeared or smoothed out pulses.

Another example is two square pulses convolved come out as a flattened trapezoid.

Think of this... Imagine a drum you are beating it repeatedly to hear the music right? Your drum stick will land on the membrane for the first time due to the impact it will vibrate , when you strikes it for the second time ,vibration due to first impact has already decayed to some extent. So whatever sound you will hear is the current beating and sum of the decayed response of previous impacts. So if $x(k)$ is the impact force on $k$ th moment, then the impact will be Force $x$ Impact time

Which is

$x(k)dk$

Where $dk$ is infinitesimaly small time of impact

and you are hearing the sound @ $t$ , then the elapsed time will be $t-k$ , suppose if the membrane of the drum has a decay effect , defined by a function $h(u)$ , where $u$ is elapsed time, in our case $t-k$ , so the response of impact @ $k$ will be $h(t-k)$. So the effect of $x(k)dk$ at time t will be multiplication of both, i.e. $x(k)h(t-k)dk$.

So the overall effect of the music we hear will be the integrated effect of all the impacts. That too from negative infinity to plus infinity. Which is what is know as convolution.

  • interesting illustration :) – tomab Aug 10 '16 at 16:44

If you take a picture with a camera with the lens defocused, the result is a convolution of the focused image with the point spread function of the defocus.

The probability distribution of the sum of a pair of dice is the convolution of the probability distributions of the individual dice.

Long multiplication is convolution, if you don't carry from one digit to the next. And if you flip one of the numbers. {2, 3, 7} convolved with {9, 4} is {8, 30, 55, 63}

      2   3   7
   X      4   9
---------------
     18  27  63 
  8  12  28
---------------
  8  30  55  63

(You could finish out the multiplication by carrying the "6" from 63 into the 55, and so on.)

In signals and systems, convolution is usually used with input signal and impulse response to get an output signal(third signal). It's easier to see convolution as "weighted sum of past inputs" because past signals also influence current output.

I'm not sure if this is the answer you were looking for, but I made a video on it recently because it bothered me for a long time. https://www.youtube.com/watch?v=1Y8wHa3fCKs&t=14s Here's a short video. Please excuse my English lol.

Another way to look at convolution is to consider that you have two things:

  • DATA - quantities certainly corrupted by some noise - and at random positions (in time, space, name it)
  • PATTERN = some knowledge of how information should look like

the convolution of DATA with (the mirror symmetric of the) PATTERN is another quantity that evaluates -knowing the PATTERN- how likely it is that it is at each of the positions within the DATA.

Technically, at every position, this quantity is the correlation (this is the mirror of the PATTERN) and thus measures the log-likelihood under some general assumptions (independent Gaussian noise). The convolution allows to compute it at each position (in space, time ...) in parallel.

A convolution is an integral that expresses the amount of overlap of one function (say $g$) as it shifted over another function ( say $f$) where $g*f$.

The physical meaning is a signal passes through an LTI system! Convolution is defined as flip (one of the signals), shift, multiply and sum. I am going to explain my intuition about each.

1. Why we flip one of the signals in convolution, What does it mean?

Because the last point in the representation of the input signal actually is the first which enters the system (notice the time axis). Convolution is defined for Linear-Timer Invariant systems. It is all related to Time and how we represent it in math. There are two signals in convolution, one represents the input signal and one represent the system response. So the first question here is What is the signal of system response? System response is the output of the system in a given time t to an input with only one non-zero element in a given time t (impulse signal which is shifted by t).

2. Why the signals are multiplied point by point?

Again, lets refer to the definition of signal of system response. As said, it is the signal which is formed through shifting an impulse function by t and plotting the output for each of these t's. We can also imagine the input signal as sum of impulse functions with different amplitudes (scales) and phases. OK, so the system response to the input signal in any given time is the signal response itself multiplied by (or scaled by) the amplitude of the input in that given time.

3. What does shifting mean?

Having said those (1 & 2), shifting is performed to get the output of the system for any input signal point at an time t.

I hope it helps you folks!

[As the question keeps bumping, a short edit] The output is the joint filtering of the two input signals or functions. In other words, how $x_1$ is smoothed by $x_2$ considered as a filter, and symmetrically how $x_2$ is smoothed by $x_1$ considered as a smoothing function. To some extent, this convolution is a kind of "Least common multiple" between two signals (instead of numbers).

A longer "system view" follows: Think of an ideal (Platonist) vision of a point. The head of a pin, very thin, somewhere in the empty space. You can abstract it like a Dirac (discrete or continuous).

A point in space

Look at it from afar, or like a short-sighted person (as I am), it gets blurred. Now imagine the point is looking at you, too. From the point "point of view", you can be a singularity, too. The point can be short-sighted as well, and the medium between you both (you as a singularity and the point) can be non-transparent.

So, convolution is like A bridge over troubled water. I never thought I could quote Simon and Garfunkel here. Two phenomena trying to seize each other. The result is the blur of one blurred by the other, symmetrically. The blurs don't have to be the same. Your short-sighted blurring combines evenly with the fuzziness of the object. The symmetry is such that if the fuzziness of the object becomes your eye-impairment, and vice-versa, the overall blur remains the same. If one of them is ideal, the other is untouched. If you can see perfectly, you see the exact blurriness of the object. If the object is a perfect point, one gets the exact measure of your short-sightedness.

All that under some linearity assumptions. The convolution is a complicated operation. In the Fourier domain, you can interpret it as a product of blurs. Or in the $\log$-Fourier domain, it can be interpreted as a sum of blurs.

You can check But Why? Intuitive Mathematics: Convolution

blur

The way you hear sound in a given environment (room, open space etc) is a convolution of audio signal with the impulse response of that environment.

In this case the impulse response represents the characteristics of the environment like audio reflections, delay and speed of audio which varies with temperature.

To rephrase the answers:

For signal processing it is the weighted sum of the past into the present. Typically one term is the voltage history at an input to a filter and the other term is the a filter or some such that has "memory". Of course in video processing all of the adjacent pixels take the place of "past".

For probability it is a cross probability for an event given other events; the number of ways to get a 7 in craps is the chance of getting a: 6 and 1, 3 and 4, 2 and 5. i.e. the sum of probabilities P(2) times the probability P(7-2): P(7-2)P(2)+P(7-1)*P(1)+.....

Convolution is a mathematical way of combing two signals to form a third signal. It is one of the most important techniques in DSP… why? Because using this mathematical operation you can extract the system impulse response. If you do not know why system impulse response is important, read about it in http://www.dspguide.com/ch6.htm. Using the strategy of impulse decomposition, systems are described by a signal called impulse response. Convolution is important because it relates the three signals of interest: the input signal, the output signal, and the impulse response. It is a formal mathematical operation, just as multiplication, addition, and integration. Addition takes two numbers and produces a third number, while convolution takes two signals and produces a third signal. In linear systems, convolution is used to describe the relationship between three signals of interest: the input signal, the impulse response, and the output signal (from Steven W. Smith). Again, this is highly bound to the concept of impulse response which you need to read about it.

Impulse causes output sequence which captures the dynamics of the system ( future). By flipping over this impulse response we use it to calculate the output from The weighted combination of all previous input values. This is an amazing duality.

In simple terms it means to transfer inputs from one domain to another domain where we find it easier to work with. Convulation is tied with Laplace transform, and sometimes it is easier to work in the s domain, where we can do basic additions to the frequencies. and also as laplace transform is a one to one function we are most likely to not corrupt the input. Before trying to understand what the general theorem of convulation means in physical significance, we should instead start at the frequency domain. addition and scalar multiplication follows the same rule as Laplace transform is a linear operator. c1.Lap(f(x)+ c2.Lap g(x)= Lap (c1.f(x) + c2.g(x)). But what is Lap f(x).Lap g(x). is what defines the convulation theorem.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.