How can convolution be a linear and invariant operation?

Question

I'm having a slight breakdown right now with a seemingly simple question. Say I have a system that convolves an input function with itself to produce an output function:

$g(x) = f(x) ∗ f(x)$

I've heard countless times that convolution is a linear operation and just assumed it was fact. However, I'm trying to understand how this is possible.

In my system above, it doesn't seem like this system could be linear since convolution contains multiplication. So multiplying an input signal with itself can't possibly be linear. However, it would appear to be shift invariant.

Am I correct in my logic? I seem to have a misunderstanding of convolution being a linear operation.

Answer 1

Convolution of an input signal with a fixed impulse response is a linear operation. However, if the input-output relation of a system is

$$y(t)=(x*x)(t)\tag{1}$$

then the system is non-linear, which is straightforward to show. Similarly, any convolution with a kernel that depends on the input signal is a non-linear operation.

On the other hand, a system with input-output relation

$$y(t)=(x*h)(t)\tag{2}$$

is linear (and time-invariant) because it convolves any input signal $x(t)$ with a fixed impulse response $h(t)$, which is independent of the input signal.

Answer 2

Your problem comes about because you're pondering the convolution of a signal by itself: $$g(x) = f(x) * f(x)$$

So, yes, if you had a block diagram where you collect f(x) from the æther and then somehow magically convolve it by itself (which can't be done physically because it would require time-reversing $f(x)$, which requires looking into the future), then yes, that would be nonlinear.

But that's not what convolution is about

Convolution is about taking an actual signal (probably what you mean by your $f(x)$), and running it through a system, who's behavior is defined by an impulse response. An impulse response is, in a sense, a signal, but it's not an actual physical signal -- it's a full and complete description of how a linear, time-invariant system behaves in response to a signal.

Here's a fairly far-fetched analogy, that isn't coming out nearly as simple and clear as I'd like*: say you live on a farm, where goats weigh 1/3 as much as sheep, and you have a scale that reads out in goat-weights. If you put three sheep on the scale, then the number you read is $3 * 3 = 9$. What it looks like you have done is multiply three by three (a nonlinear operation if $3$ is a signal) and gotten 9. What you have actually done is multiply a signal (three sheep-weights) by a fixed characteristic of a signal (one estimated-goat-weight / actual-goat-weight) and gotten an output. Had you put four sheep on the scale, only one part of your calculation would change -- the "three goats per sheep" part would still be a three.

So in your original example, you're trying to convolve a signal "by itself". That doesn't happen. You may convolve a signal by an impulse response that happens to be equal to that signal -- and even I, in a classroom setting, may tell the class that I'm "convolving the signal by itself", only to have to make the above clarification.

You'll have occasion to find the autocorrelation of a signal, which looks a lot like convolution (it looks like $A_f(x) = f(x) * f(-x)$, in fact) and for which you sometimes trick a math package into doing because it doesn't have a built-in autocorrelation function.

You sometimes have occasion to find out what happens when you feed a filter with a signal that happens to match its impulse response -- but then you're feeding one distinct signal into a filter that's got eerily similar characteristics to the signal**.

You may even build a system that involves delay lines and multiplications and whatnot that "convolves a signal by itself" -- but then you're building a filter on the fly that gives itself an impulse response that matches a delayed version of a collected signal, and convolving.

But you never really convolve a signal by itself.

* I should never, ever, get on these boards before the caffeine has kicked in. Yet, I regularly do.

** Well, probably quite intentionally similar characteristics, but -- caffeine.

Answer 3

Standard multiplication by a scalar $s$ is a linear operation, since:

$$ s\times(a\times x+b\times y)= a\times (s\times x)+ b\times(s\times y)$$

In other words, multiplying a linear combination by a scalar is the same as doing a linear combination of multiplies. Here, the multiplication is a very simple example of a system(a amplifier).

Your question amounts to wonder whether the multiplication is not linear, because $x\times x$ is quadratic. A system which would convolve a signal by itself would obviously be non linear, but the underlying operation can still be.