From the high school maths we know that y=mx+c is a linear equation. However, in DSP the linear system must satisfy Additivity properties which y=mx+c does not hold because of +c. So, is the definition of Linearity different in DSP and Maths? if so why? Thanks.


Yes, in some sense the definitions are different. I'll give you two points of view, the first one will support your keen observation, the other will provide evidence for the contrary. These two do not conflict with each other, it's a matter of semantics. If the second one confuses you stick with the first one. Prelude concluded, here goes.

Problem Definition 1: $y = mx + c $ means $f(x)=mx+c$

Here we take the usual point of view in which $y$ is assumed to be the output of some manipulations on $x$. We call that something a function, and we can write down the same expression with a little more mathematical elegance: $f(x)=mx+c$. It is now clear that $f(x)$ is in some sense on output of some mathematical manipulation to which $x$ is the input.

Let's refresh criteria for linearity. A function $g(x)$ is linear is it satisfies both of the following conditions:

  1. $g(a+b) = g(a)+g(b)$ for all $a$ and $b$
  2. $g(cx) = cg(x)$ for all constants $c$

Clearly, our favorite function $f(x)$ does not satisfy either of these properties. So yes, from this perspective $f(x)$ is not a linear function. The closest thing to "linear" we can call it is "affine".


You may now brace yourself for part 2 of the answer.

Problem Definition 2: $y = mx + c $ means $L(x,y)=y-mx$

Let's take it one step at a time. Suppose you're trying to solve a system of two linear equations. How do you do it? One way is to write down the equations as follows:

$$ \begin{align} y &= m_1x + c_1\\y &= m_2x + c_2 \end{align}$$

Surely, that's how we've all been doing it since seventh grade. Now all you have to do is solve it by substitution or whichever way you prefer. But what do you do when you have a system of equations of more that two variables? Will you write it down like so?

$$ \begin{align} y &= a_1x + c_1z + d_1\\y &= a_2x + c_2z + d_2\\y &= a_3x + c_3z + d_3 \end{align}$$

That doesn't really look right. And for a very good reason. There are many ways of interpreting functions of any number of variables, and it's not just the semantics that's different. To digress for a moment, take the equation $x^2+y^2=r^2$. Almost anyone (visiting this forum that is) will immediately identify it as an equation of a circle. But recall definition of a function!

If we interpret it as $f(x) = \pm \sqrt{r^2-x^2}$ we get two solutions: an upper half of a circle and a lower half of a circle. The entire circle cannot be a solution because it violates the property that in functions, for every input there's at most one unique output.

If we on the other hand interpret it as $f(x,y)=r^2$, we get back the entire circle as a solution, because we're viewing it as a function of two variables equal to a constant. In other words, even though we're written the same expression $x^2+y^2=r^2$, we must define what we're talking about. Otherwise this problem is not well-defined. In one interpretation it is a function $f:\mathbb{R} \rightarrow \mathbb{R}$, in another interpretation it is a function $f:\mathbb{R^2} \rightarrow \mathbb{R}$. Remember all that mumble about domains and ranges in high school? Yeah, this is exactly what it is. Now, back to our myterious topic of linear functions.

Hopefully, by this point you're already has your aha! moment. If not, here's our finish straight. Remember that system of three equations that didn't quite look right? First of all note that it looks affine, because in addition to variables $x$ and $z$ there are constants $d$ as well. Now a nicer way of writing down this system of equations is like so:

$$ \begin{align} -a_1x + y + -c_1z &= d_1 \\ -a_2x + y + -c_2z &= d_2 \\ -a_3x + y + -c_3z &= d_3\end{align}$$

Now we're getting somewhere. As you can see, we can write it out in matrix form as follows:

$$\begin{bmatrix} -a_1 &1 &-c_1 \\-a_2 &1 &-c_2 \\-a_3 &1 &-c_3 \end{bmatrix}\begin{bmatrix} x \\ y \\ z \end{bmatrix} = \begin{bmatrix} d_1 \\ d_2 \\ d_3 \end{bmatrix}$$

Clearly, this is a linear system of equations. Where's the catch? Well, at first it looked like a system of three functions of the form $f:\mathbb{R^2} \rightarrow \mathbb{R}$, and now we're representing it as a single function of the form $f:\mathbb{R^3} \rightarrow \mathbb{R^3}$.

To clarify, this is a single function which takes in a vector in $\mathbb{R}^3$ and returns another vector in $\mathbb{R}^3$. Let's call this function $L(x,y,z)$, precicely $L:\mathbb{R^3} \rightarrow \mathbb{R^3}$. I will let you check that this function is linear. Concretely, if $\begin{bmatrix} a_{11} &a_{12} &a_{13} \\a_{21} &a_{22} &a_{23} \\a_{31} &a_{32} &a_{33} \end{bmatrix}\begin{bmatrix} x \\ y \\ z \end{bmatrix} = \begin{bmatrix} d_1 \\ d_2 \\ d_3 \end{bmatrix}$ and $\begin{bmatrix} a_{11} &a_{12} &a_{13} \\a_{21} &a_{22} &a_{23} \\a_{31} &a_{32} &a_{33} \end{bmatrix}\begin{bmatrix} \alpha \\ \beta \\ \gamma \end{bmatrix} = \begin{bmatrix} \delta_1 \\ \delta_2 \\ \delta_3 \end{bmatrix}$, then

  1. $\begin{bmatrix} a_{11} &a_{12} &a_{13} \\a_{21} &a_{22} &a_{23} \\a_{31} &a_{32} &a_{33} \end{bmatrix}\begin{bmatrix} x+\alpha \\ y+\beta \\ z+\gamma \end{bmatrix} = \begin{bmatrix} d_1 + \delta_1 \\ d_2 + \delta_2 \\ d_3 + \delta_3 \end{bmatrix}$, and
  2. $\begin{bmatrix} a_{11} &a_{12} &a_{13} \\a_{21} &a_{22} &a_{23} \\a_{31} &a_{32} &a_{33} \end{bmatrix}\begin{bmatrix} kx \\ ky \\ kz \end{bmatrix} = \begin{bmatrix} kd_1 \\ kd_2 \\ kd_3 \end{bmatrix}$

In other words (and yes, this is the real reason mathematicians keep constantly coming up with new concise notation!), let $\vec{u},\vec{v} \in \mathbb{R}^3$ ($\vec{u}$ and $\vec{v}$ and 3-dimensional vectors of real numbers). Then

  1. $L(\vec{u}+\vec{v}) = L(\vec{u})+L(\vec{v})$
  2. $L(k\vec{u}) = kL(\vec{u})$

Linear! Q.E.D.

In conclusion, we've explored mysterious subtleties of mathematics of functions and in particular the importance of defining problems well. The function $f(x) = mx + c$ is obviously non-linear (or more precisely affine), and the function $g(x,y) = y - mx$ is linear.

Come back for more interesting stuff. We like giving twisted answers to simple questions.

| improve this answer | |
  • 2
    $\begingroup$ A simpler reason for calling $y=mx+c$ a linear function is that its graph is a straight line not necessarily through the origin, and at least in the elementary analytical geometry that most readers will have encountered in high school and first heard the name, it is far too complicated to have different names for straight lines that pass through the origin and those that don't. $\endgroup$ – Dilip Sarwate Feb 5 '13 at 13:13
  • $\begingroup$ Wikipedia says this as Linear polynomials... I wonder why dont DSP books include this information! $\endgroup$ – Zahid Hasan Feb 5 '13 at 16:53
  • $\begingroup$ @Phonon - Excellent presentation. This is the second stack exchange answer I would like to keep in my archives for future reference. $\endgroup$ – user2718 Feb 5 '13 at 18:34
  • 1
    $\begingroup$ @BruceZenone Over on math.SE, they maintain a question List of Generalizations of Common Questions on their meta site which has a collection of best answers that are saved for future reference. Perhaps a similar thing could be added to dsp.SE (Hint, hint, moderator Phonon!) and would help answer questions such as What constitutes a FAQ question on the dsp meta site. $\endgroup$ – Dilip Sarwate Feb 6 '13 at 14:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.