9
$\begingroup$

I am looking for a proof for sinusoidal fidelity. In DSP we study a lot about linear systems. Linear systems are homogenous and additive. One more condition it satisifies is that if a signal is a sine or cos wave then the output only changes the phase or amplitude. Why? Why cant the output be a totally different output when a sine wave is given as an input?

$\endgroup$
  • 1
    $\begingroup$ Welcome to DSP. Great question! $\endgroup$ – Phonon Jul 18 '12 at 13:44
  • 5
    $\begingroup$ Your understanding is incomplete. A linear (meaning homogeneous and additive) system does not necessarily have the property that an input sinusoid produces a sinusoid of the same frequency but possibly different amplitude and phase. You need to impose the further restriction that the system is also time-invariant. For example, if input $x(t)$ produces output $x(t)\cos(2\pi 10^9 t)$, the system is homogeneous and additive, and hence linear, but does not satisfy the SISO (sinusoid in- sinusoid out) property. $\endgroup$ – Dilip Sarwate Jul 18 '12 at 14:00
  • $\begingroup$ Dilip (or someone) should put as an answer: "They don't ." Only time-invariant linear systems do. $\endgroup$ – hotpaw2 Jul 18 '12 at 18:33
  • 2
    $\begingroup$ Just as a note, another way to phrase this question would be "Why are exponentials eigenfunctions of linear time-invariant systems?" $\endgroup$ – Jason R Jul 19 '12 at 13:25
8
$\begingroup$

A somewhat visual complement to the other answers

You are talking about systems that are linear and time invariant.

Exponential functions have one peculiar property (and can be actually defined by it): doing a time translation results in the same function multiplied by a constant. So

$$ e^{t-t_0}=e^{-t_0}e^t$$

Mathematica graphics

The red exponential could as well be the blue one divided by $e$ or moved 1 second to the right

In general, this also holds for complex exponentials

Can you picture in your mind the plot of an complex harmonic such as $x(t)=e^{j2\pi t}$? If so, you will see it's like a spring: it rotates along the complex plane as time goes by.

Mathematica graphics

Rotating that spring (multiplying by a complex number in the unit circle) is the same as translating it. You probably have come into this visual effect some time in your life

enter image description here

It's the principle of any standard screw too.

Assume we input this in a linear time-invariant system. You get an output $y$ Now, input a rotated version of this spring. Because of linearity, the output should be $y$ rotated by the same amount. But since a rotation is equivalent to a time-translation, and the system is time-invariant, the output also has to be $y$ time-translated by the same amount. So, $y$ has to satisfy the same property as the input: rotating it has to be equivalent to a particular time translation. This only happens when the output is a multiple of the original spring.

How much translation? Well, it's directly proportional to the rotation just like it would happen with a spring. The tighter the loops of the spring (the faster it rotates), the less it time-translates for a certain rotation. The tighter the loops of a screw, the more rounds you have to make it do for it to fit completely. And, when half of the rounds are done, the screw will be half it's way in... The output has to satisfy the same relation so, the output spring $y$ rotates at the same frequency as the input.

At last, a reminder

$$\cos(t)=\frac{e^{j t}+e^{-j t}}{2}$$

$$\sin(t)=\frac{e^{j t}-e^{-j t}}{2 j}$$

So, that thing that happens with exponentials actually doesn't need to happen with cosines and sines in the most general case. But if the system is also real it's a different story...

In general, by this same reasoning, any exponential is an "eigenfunction" (output is proportional to input) of linear time invariant systems. That's why for these systems Z-transforms and Laplace transforms are so useful

$\endgroup$
  • $\begingroup$ How/Where did you get that animation from? $\endgroup$ – Spacey Jul 19 '12 at 15:06
  • $\begingroup$ @Mohammad took it from the wikipedia page on Archimedes screw $\endgroup$ – Rojo Jul 19 '12 at 15:39
  • $\begingroup$ Where did you get that corkscrew plot? :) math.stackexchange.com/q/144268/2206 $\endgroup$ – endolith Jul 25 '12 at 19:59
  • $\begingroup$ @endolith oh I just did it in Mathematica. Yours are nicer ;) $\endgroup$ – Rojo Jul 25 '12 at 20:31
4
$\begingroup$

Consider a system with input $x(t)$ and output $y(t)$. Borrowing notation from Lars1's answer, we denote this relationship $x(t) \to y(t)$. The system is said to be a linear time-invariant (LTI) system if it satisfies the following properties:

H. If $x(t) \to y(t)$, then $\alpha x(t) \to \alpha y(t)$.

A. If $x_1(t) \to y_1(t)$ and $x_2(t) \to y_2(t)$, then $x_1(t) + x_2(t) \to y_1(t) + y_2(t).$

T. If $x(t) \to y(t)$, then $x(t-\tau) \to y(t-\tau)$ for any real number $\tau$.

Properties H and A together are equivalent to Property L

L. If $x_1(t) \to y_1(t)$ and $x_2(t) \to y_2(t)$, then $\alpha x_1(t) + \beta x_2(t) \to \alpha y_1(t) + \beta y_2(t)$.


Periodic input to a time-invariant system produces periodic output
Suppose that $x(t)$ is a periodic signal with period $T$, that is, $x(t-nT) = x(t)$ for all integers $n$. Then, from Property T, it follows immediately that $y(t)$ is also a periodic signal with period $T$. Thus, we can express $y(t)$ as a Fourier series:

$$y(t) = \frac{a_0}{2} + \sum_{n=1}^{\infty} a_n \cos(n\omega t) + b_n \sin(n\omega t)$$ where $\omega = 2\pi/T$ is the fundamental frequency.


Since $\cos(\omega t)$ and $\sin(\omega t)$ are periodic signals, we have that for any time-invariant system, whether linear or not, $$\begin{align*} \cos(\omega t) &\to \frac{p_0}{2} + \sum_{n=1}^{\infty} p_n \cos(n\omega t) + q_n \sin(n\omega t)\\ \sin(\omega t) &\to \frac{r_0}{2} + \sum_{n=1}^{\infty} r_n \cos(n\omega t) + s_n \sin(n\omega t)\\ \end{align*}.$$ In fact, for linear time-invariant (LTI) systems, all the $p_n, q_n, r_n,$ and $s_n$ are zero except for $p_1, q_1, r_1, s_1$. To see why this is so, let us compute the LTI system's response to $\cos(\omega t - \theta)$ in two different ways and compare the results.

Since $\cos(\omega t - \theta) = \cos(\theta)\cos(\omega t) + \sin(\theta)\sin(\omega t)$, we get from Property L and the above equations that $$\begin{align*} \cos(\omega t - \theta) &\to \frac{p_0\cos(\theta) + q_0\sin(\theta)}{2}\\ &\qquad + \sum_{n=1}^{\infty} (p_n\cos(\theta) + r_n\sin(\theta))\cos(n\omega t)\\ &\qquad + \sum_{n=1}^{\infty} (q_n\cos(\theta) + s_n\sin(\theta))\sin(n\omega t). \end{align*}$$ On the other hand, since $\cos(\omega t - \theta) = \cos(\omega (t-\theta/\omega))$ is just a delayed version of $\cos(\omega t)$, from Property T we get that $$\begin{align*} \cos(\omega t - \theta) &\to \frac{p_0}{2} + \sum_{n=1}^{\infty} p_n \cos(n\omega t -n\theta) + q_n \sin(n\omega t - n\theta)\\ &= \frac{p_0}{2} \\ &\qquad + \sum_{n=1}^{\infty} (p_n\cos(n\theta) - q_n\sin(n\theta))\cos(n\omega t)\\ &\qquad + \sum_{n=1}^{\infty} (q_n\cos(n\theta) + p_n\sin(n\theta))\sin(n\omega t) \end{align*}.$$ These two Fourier series must be the same no matter what value of $\theta$ we choose. Comparing coefficients, we see that $p_0/2$ cannot equal $(p_0\cos(\theta) + r_0\cos(\theta))/2$ for all $\theta$ unless $p_0 = r_0 = 0$. Similarly, for any $n > 1$, $p_n\cos(n\theta) - q_n\sin(n\theta)$ cannot equal $p_n \cos(\theta) + r_n\sin(\theta)$ etc. for all $\theta$ unless $p_n = q_n = r_n = s_n = 0$. However, for $n=1$, $p_1\cos(\theta) - q_1\sin(\theta) = p_1\cos(\theta) + r_1\sin(\theta)$ implies that $r_1 = -q_1$, and similarly, $s_1 = p_1$. In other words, for an LTI system, $$\begin{align*} \cos(\omega t) &\to p_1 \cos(\omega t) + q_1\sin(\omega t)\\ \sin(\omega t) &\to -q_1 \cos(\omega t)+ p_1 \sin(\omega t)\\ \end{align*}.$$ Now, $p_1 \cos(\omega t) + q_1\sin(\omega t) = B\cos(\omega t - \phi)$ where $B = \sqrt{p_1^2+q_1^2}$ and $\phi = \arctan(q_1/p_1)$. Therefore, Properties T and H give us that $$A\cos(\omega t - \theta) \to AB\cos(\omega t - \phi - \theta).$$ Any sinusoid of frequency $\omega$ rad/s can be expressed as $A\cos(\omega t - \theta)$ for appropriate choice of $A$ and $\theta$, and so the above result is what we need.

SISO property of linear time-invariant systems: If the input to an LTI system is a sinusoid, the output is a sinusoid of the same frequency but possibly different amplitude and phase.

This is not quite the result that the OP wanted -- he wanted a proof that a linear system (one in which Properties H and A (equivalently, Property L) hold but not necessarily Property T) has the SISO property, but as the development above shows, Property T must hold in order to prove even the weaker result that periodic input results in periodic output.


As a final comment, note that it is not necessary to use complex numbers or convolution theorems or Fourier or LaPlace transforms, impulses, eigenfunctions etc to prove the SISO property. It follows from Properties L and *T and the trigonometric identity $$ \cos(\alpha - \beta) = \cos(\alpha)\cos(\beta) + \sin(\alpha)\sin(\beta).$$

$\endgroup$
  • $\begingroup$ What would happen if $x(t)$ is not periodic (not periodic could happen for incommensurate frequencies)? Need $T$ be finite? Could we gain something in terms of generality by requiring $x(t)$ to be square integrable in the observation time interval? $\endgroup$ – Lars1 Jul 22 '12 at 7:21
  • $\begingroup$ @Lars1 If the input to a LTI system is not periodic, the output is not periodic either. As a specific case, if $x(t) = A_1\cos(\omega_1 t) + \A_2\cos(\omega_2 t)$ where $\omega_1/\omega_2$ is irrational (and so input is not periodic), then from Property L we have that $$A_1\cos(\omega_1 t) + \A_2\cos(\omega_2 t) \to A_1 B_1\cos(\omega_1 t - \phi_1) + \A_2B_2\cos(\omega_2 t-\phi_2)$$ which output is not periodic either. So there is no problem. $\endgroup$ – Dilip Sarwate Jul 22 '12 at 22:08
  • $\begingroup$ @Sarwate: Not quite what I meant to say, sorry. Was wondering if e.g. $x(t)=\cos(\pi t)+\cos(\sqrt{2}t)$ would be handled by the case above. If we require a finite observation time interval with $t\in\mathbb{T}=[0;T]$ any square integrable signal can be written as a Fourier series in the observation interval. For finite $T$ this is likely the most general approach and your derivations still hold as far as I can see. Obviously the Fourier series approach forces periodicity outside $\mathbb{T}$ but if we only care about the signal $t\on\mathbb{t}$ this does not really matter. $\endgroup$ – Lars1 Jul 23 '12 at 10:10
  • $\begingroup$ @Lars1 I don't agree with your comment that the enforced periodicity outside $[0, T]$ does not matter. If input $x(t)$ produces output $y(t)$ in an LTI system, then applying the SISO property to the Fourier series does not give $y(t)$ restricted to $[0,T]$. Instead, what is obtained is one period of the periodic response $\hat{y}(t)$ to the periodic signal $\hat{x}(t)$ where for each time instant $t$, $-\infty< t < \infty$, $$\hat{x}(t) = x(t \bmod T).$$ In other words, the $T$-second segment of $x(t)$ repeated periodically (with period $T$) along the time axis. $\endgroup$ – Dilip Sarwate Jul 23 '12 at 20:03
  • $\begingroup$ E.g. in nonlinear RF systems we often choose a sum of incommensurate sinusoidals to ensure a unique frequency mapping from input to output. These result in a non-periodic signal, and I just was curious to why you had to assume periodicity above which to me seems to exclude most practically relevant signals. Square integrable $x(t)$ and $y(\tau)$ in finite observation intervals can be written as Fourier series. I did not (intend to) claim that $t$ was defined on the same interval for $x$ and $y$ BTW and $y$ could be a time offset version. I'll stop here to avoid further confusion. $\endgroup$ – Lars1 Jul 25 '12 at 13:39
3
$\begingroup$

Here's the idea of the proof. Let's assume we can describe the output of a system by a convolution,

$$y(t) = \int k_t(t-\tau) f(\tau) d\tau$$

Notice that the function (aka "kernel") $k_t(t)$ as I've written it here may change as $t$ varies. However, we usually make an important assumption about $k_t(t)$ - that it doesn't change with time. This is called "linear time-invariance" (also check out the Wikipedia page on Toeplitz matrices). If our system is linear time-invariant, $k_t$ is the same for any $t$, and so we'll just ignore the subscript and write

$$y(t) = \int k(t-\tau) f(\tau) d\tau$$

Now, let's say $f(t)$ is a sinusoid, say $f(t) = e^{i\omega t}$. So, we have

$$ y(t) = \int k(t-\tau) e^{i\omega \tau} d\tau = \int k(\tau) e^{i\omega (t-\tau)} d\tau = e^{i\omega t} \int k(\tau) e^{-i\omega \tau}d\tau $$

Notice that the last equation has no dependence on $t$! As a result, let's define $K(\omega) := \int k(\tau) e^{-i\omega \tau}d\tau$.

Thus, we've discovered that

$$ y(t) = K(\omega) e^{i\omega t} $$

or, in other words, $y(t)$ is a sinusoid oscillating at the same frequency as the input, but weighted by a complex number $K(\omega)$ which is constant with respect to $t$ (and thus may shift the amplitude and phase of the output with respect to the input).

EDIT: The comments noted this answer was pretty loose. My goal was to avoid details like different forms of the Fourier transform, but I ended up conflating the Fourier and Laplace transforms. What I called Fourier transform previously was only the Fourier transform if $s$ was purely imaginary. I decided that clarifying this route would necessarily add too much notation, so I'm relegating it to italics.

Now, take the Laplace transform, to end up with (since Laplace transform takes convolution to multiplication),

$$Y(s) = K(s) F(s)$$

Now, if $f$ is a sinusoid, say $f(t)=e^{i\omega t}$, its Laplace transform is a delta function at that $\omega$. That is, $F(s) = \delta_{w}(s)$. So, the Laplace transform of the output is also a delta function at that frequency:

$$ Y(s) = K(s)\delta_\omega (s) = K(\omega) \delta_\omega(s)$$

Since $K(\omega)$ is just some complex number that depends on the input frequency, the output $y(t)$ will be a sinusoid with the same frequency as the input, but with potentially different amplitude and phase.

Incidentally, I just noticed you can find the same idea written out in the time domain at Wikipedia. A higher-level explanation (which you can ignore if it's too mathy) is that linear systems theory is defined through the convolution operation, which is diagonalized by the Fourier transform. Thus, a system whose input is an eigenvector of the Fourier transform operator will output only a scaled version of its input.

$\endgroup$
  • $\begingroup$ -1 What is $s$ and how does it relate to $\omega$? And could you explain what is meant by $\delta_{\omega}(s)$? Your equation $Y(s) = K(s)\delta_{\omega} s)$ is sheer nonsense. $\endgroup$ – Dilip Sarwate Jul 18 '12 at 12:26
  • $\begingroup$ @DilipSarwate I suspect he's using Laplace transform notation instead of Fourier notation. $\endgroup$ – Jim Clay Jul 18 '12 at 13:23
  • $\begingroup$ @sydeulissie The problem is that you assert that K(w) is "just some complex number", but you haven't said why it's just a complex number at each frequency. That's the heart of the proof. $\endgroup$ – Jim Clay Jul 18 '12 at 13:24
  • 3
    $\begingroup$ This has a correct outline but many problems in the details. Not downvoting, but it should be fixed. $\endgroup$ – Phonon Jul 18 '12 at 13:47
1
$\begingroup$

Say we have a system with input $x_1(t)$ which generates the output $y_1(t) = {\cal G}(x_1(t))$, and with an input $x_2(t)$ we get the output $y_2(t) = {\cal G}(x_1(t))$. The system is linear if:

$$ a\cdot x_1(t) + b\cdot x_2(t) \rightarrow y(t) = {\cal G}(a\cdot x_1(t) + b\cdot x_2(t)) = a\cdot {\cal G}(x_1(t)) + b\cdot {\cal G}(x_2(t)) = a\cdot y_1(t) + b\cdot y_2(t) $$

where $a$ and $b$ are (real or complex) constants. If the equations above are not fulfilled the system is nonlinear. The equation can be used for real and complex signals in time and frequency domains. This is the same as the superposition principle must be valid. As Sarwate illustrates in a comment this does not prevent the system from generating new frequencies. We are probably often just used to indirectly assume time invariance. The reason is likely that it is often possible to map a time varying system to a time invariant system by applying one or more external controlling signals.

From the definition of linearity and further requiring a time invariant system we can directly see that two (or more signals) can not interfere and generate new frequency components while still complying with the linearity requirement. The principle of superposition also follows directly from the linearity definition.

Also from the linearity definition the concept of convolution for linear time invariant systems follow. For nonlinear systems we for example have Volterra series which is a multi-dimensional convolution integral - the 1-dimensional convolution integral is a special case of the Volterra series. This is way more complicated than linear techniques though. But based on the convolution integral for a linear system the derivation follows the one shown by @sydeulissie.

To demonstrate a simple counter example of a nonlinear relation where new frequencies are generated we could use ${\cal G}: y(t) = x^2(t)$. Let us first show that this is indeed nonlinear. If we apply the input $x_1(t)$ we get the output $y_1(t) = x_1^2(t)$ and if we apply the input $x_2(t)$ we get the output $y_2(t) = x_2^2(t)$. The output $y(t)$ is then:

$$ y(t) = \left\{ a\cdot x_1(t) + b\cdot x_2(t) \right\}^2 = a^2\cdot x_1^2(t) + b^2\cdot x_2^2(t) + 2\cdot a\cdot b \cdot x_1(t) \cdot x_2(t) $$

or:

$$ y(t) = a^2 \cdot y_1(t) + b^2 \cdot y_2(t) \pm 2\cdot a \cdot b \cdot \sqrt{y_1(t) \cdot y_2(t)} \neq a\cdot y_1(t) + b\cdot y_2(t) $$

and we have thus proved $x^2$ to be nonlinear (which can hardly be surprising). If we apply a single sinusoidal signal $x(t) = A\cdot \cos(2\pi f_0 t + \phi_0)$ to the system ${\cal G}$ we have the output:

$$ y(t) = x^2(t) = A^2 \cdot \cos^2(2\pi f_0 t + \phi_0) = \frac{A^2}{2} + \frac{A^2}{2}\cdot \cos(2\pi 2 f_0 t + 2\phi_0) $$

The output here contains a DC component and another component at the frequency $2f_0$. The nonlinear function $x^2$ thus generates new frequency components.

In conclusion it can be observed that a linear system may generate frequency components not present in the input (if the system is time variant). If the system is linear time invariant the output can not include frequency components not present in the input.

Thanks to @Sarwate for the most relevant comment.

$\endgroup$
  • $\begingroup$ You are right. I forgot to mention that I refer to time invariant systems. The example you provide is a time varying system where your example does not hold. Normally such a signal as the $\cos(t)$ is applied at an external port as a signal in which case the linearity is not fulfilled. I have noted the time invariant part in the answer above. $\endgroup$ – Lars1 Jul 18 '12 at 16:47
  • $\begingroup$ @DilipSarwate So is that that only LTI systems have that property? $\endgroup$ – Phonon Jul 18 '12 at 16:53
  • $\begingroup$ Just checked a couple of books to be on the safe side. Actually there seems to be some difference in the details. One definition in Yang and Lee's book on circuit systems from 2007 says: "A system is said to be linear if the superposition principle holds, i.e. its output to a linear combination of several arbitrary inputs is the same as the linear combination of the outputs to individual inputs". In that respect Sarwate's example is linear - but not time invariant. Other refs are less precise though. Thanks to @Sarwate. $\endgroup$ – Lars1 Jul 18 '12 at 17:28
  • 1
    $\begingroup$ Comment referred to by Lars1 with typographical errors corrected: Consider the system that produces output $x(t)\cos(t)$ from input $x(t)$. Then, $ax_1(t)+bx_2(t)$ produces output $$(ax_1(t)+bx_2(t))\cos(t)=a⋅x_1(t)\cos(t) +b⋅x_2(t)\cos(t)$$ so that the system is linear but without the claimed property. $\endgroup$ – Dilip Sarwate Jul 18 '12 at 21:04
  • $\begingroup$ @Sarwate How is the system which produces output x(t) cos(t) time varying? I am a beginner in DSP's $\endgroup$ – Hobyist Jul 18 '12 at 21:25
1
$\begingroup$

As Dilip Sarwate pointed out, only linear shift-invariant (LSIV) systems have the SISO (sinusoid in- sinusoid out) property.

The short answer to your question is that the complex exponentials $ e^{\jmath \omega t}$ are are the eigenfunctions of a LSIV system. By the definition of eigenfunction, if the input is eigenfunction (sine/cos can be represented by complex exponential according to Euler's formula), the output is just the product of the input and the corresponding eigenvalue, which could be a complex number, and that's where changes the phase/amplitude come from.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.