2
$\begingroup$

This question is perhaps related to the semantics or jargon of signal processing. I have checked plenty of advanced books/monographs on multivariable calculus and signal processing but could find an explicit answer.

Suppose we have a simple function $y$=$\sin(x)$. In short, if we have sampled a sine wave as a function of time, we will have discrete values of the sine wave corresponding to a uniform grid of time. We can list all the $y$ values in a column, and call it a signal vector.

Now how can one interpret this as a vector? Is this a point in an $n$-dimensional $\mathbb{R^n}$, which is perhaps not true for 1D discretizes signals, but certain book on Fast Fourier Transform does interpret it that way. For example, Mathematics of the Discrete Fourier Transform writes in Chapter 5,

5.2 Signals as Vectors For the DFT, all signals and spectra are length $N$. A length $N$ sequence $x$ can be denoted by $x(n), n=0,1,2, > \ldots N-1$, where $x(n)$ may be real $\left(x \in \mathbf{R}^N\right)$ or complex $\left(x \in \mathbf{C}^N\right)$. We now wish to regard $x$ as a vector ${ }^1 \underline{x}$ in an $N$ dimensional vector space. That is, each sample $x(n)$ is regarded as a coordinate in that space. A vector $\underline{x}$ is mathematically a single point in $N$-space represented by a list of coordinates $\left(x_0, x_1, x_2, \ldots, x_{N-1}\right)$ called an $N$-tuple. (The notation $x_n$ means the same thing as $x(n)$.) It can be interpreted geometrically as an arrow in $N$-space from the origin $\underline{0} \triangleq$ $(0,0, \ldots, 0)$ to the point $\underline{x} \triangleq\left(x_0, x_1, x_2, \ldots, x_{N-1}\right)$. We define the following as equivalent: $$ x \triangleq \underline{x} \triangleq x(\cdot) \triangleq\left(x_0, x_1, \ldots, x_{N-1}\right) \triangleq\left[x_0, x_1, \ldots, x_{N-1}\right] \triangleq\left[x_0 x_1 \cdots x_{N-1}\right] $$

This is exactly the same notion of vectors in vector calculus books

This notion is fine but this conceptual problem starts when one starts to think about the numerical derivative of this sine function "vector". The 1D dimensional gradient is easy to visualize since it will just be $y_{i+1} - y_i$, but if this vector is a point in $n$-dimensional space, how do we think about its derivative? I feel some important notion is missing here.

In short, how can one distinguish that this given "vector" is a discretize time based signal of one variable such as time or this signal is a vector in N-dimensional space $\mathbb{R^N}$

I have not formally studied DSP, however this interest is due to signal processing in chemical problems.

$\endgroup$
5
  • 1
    $\begingroup$ I think you may have a somewhat tilted understanding of math. It seems you have seen one instance of a derivative in a higher dimensional space and believe that is it. Mathematicians tend to define things very abstractly, in fact, trying to get to the peak of abstraction is one of the key motivations of mathematics. So don't confuse special cases with general principles. What you've likely seen in your studies has been applied math, confined to a very special situation. There's a lot more out there. $\endgroup$
    – Jazzmaniac
    Mar 13, 2023 at 22:04
  • $\begingroup$ I have nothing against these notions or abstractness. Perhaps you missed the key point of this query. The point was that when one calls a sequence of numbers as a vector, how does one distinguish that this data array is to be considered a discretized 1D function (=function of one variable), and when do you we consider that as a point in R^n. $\endgroup$
    – AChem
    Mar 14, 2023 at 4:40
  • $\begingroup$ You distinguish that by making the proper definitions and providing contextually relevant operations. In other words, you do what makes sense given the situation. Applied math tries to find the right specialisations of the abstract mathematical structures so that a meaningful model of your subject emerges. You cannot tell that from the fact that you're in a vector space alone. You're responsible for providing the additional structure. $\endgroup$
    – Jazzmaniac
    Mar 14, 2023 at 10:40
  • $\begingroup$ Right, so the label "vector" for a signal is rather utilitarian. If we want add two 1D signals, that behaves like vector addition or multiply that by a scalar, that is also fine. However, when one thinks about the numerical gradient of this "vector", it is treated like a discretized 1D array, and the gradient becomes finite difference with one variable. It is no longer considered in R^n. $\endgroup$
    – AChem
    Mar 14, 2023 at 16:04
  • $\begingroup$ Not quite. Vector spaces don't have any differential structure to begin with. If you want to do calculus on a vector space, you need to add this structure. You've encountered one incarnation of differentiation on a vector space in your applied math class, but it's far from the only one. So what is done with discrete signals ist just as valid. You're mistaken to attribute whatever you've learned about derivatives in R^n to the vector space. They are independent. $\endgroup$
    – Jazzmaniac
    Mar 14, 2023 at 18:18

2 Answers 2

2
$\begingroup$

This is exactly the same notion of vectors in vector calculus books

Yes, and in fact, the DFT maps the $\mathbb R^N$ (or $\mathbb C^N$) time-domain space to the $\mathbb C^N$ space -- and you can express that mapping as a matrix multiply.

Note, too, that any 1D vector can be taken as a function of one variable. Just let $x(k) = x_k$, and there you are.

Perhaps your difficulty stems from the fact that in signal processing we usually want to think of a signal as something evolving in time (or space), but for analysis we still often want to use the tools of Linear algebra or multivariate calculus. When we want to do the latter, we declare our signal a vector so that we can use those tools on it.

The 1D dimensional gradient is easy to visualize since it will just be $y_{i+1}−y_i$, but if this vector is a point in n-dimensional space, how do we think about its derivative?

I think your thinking may be a bit too concrete here. But you could certainly consider the derivative of $\mathbf y$ to be a mapping either $\mathbb R^N \to \mathbb R^{N-1}$ (if you want to disallow the $n^{th}$ value of the first difference) or $\mathbb R^N \to \mathbb R^N$ (if you wanted to allow it somehow, possibly by defining your first derivative as $y_{i + 1\bmod N\ -\ y_i}$).

I.e., if you define the first difference of $\mathbf y$ as $\Delta \mathbf y$, then there's a matrix $\mathbf A$ that implements $\Delta \mathbf y = \mathbf A \mathbf y$. $\mathbf A$ can be either $N \times N-1$ or $N \times N$, depending on whether you want a wrapping on non-wrapping first difference.

I will say this -- some of this signal processing stuff can be quite counter-intuitive until you've worked through a great number of examples. I remember having to go on some very long walks and very long bicycle rides to really get my head wrapped around it.

$\endgroup$
4
  • 2
    $\begingroup$ And then on the bicycle ride if you rode at a constant rate you saw the wheels as one of the basis functions in the DFT! $\endgroup$ Mar 13, 2023 at 12:15
  • $\begingroup$ OK -- (clanking noises from under the hood). Try it now. $\endgroup$
    – TimWescott
    Mar 13, 2023 at 21:38
  • 1
    $\begingroup$ @DanBoschen only if you hit a brick wall after an integer number of revolutions of the tires. I much prefer that analogy, but with the discrete-time Fourier transform -- it's much less painful. $\endgroup$
    – TimWescott
    Mar 13, 2023 at 21:44
  • 1
    $\begingroup$ Or ride in circles :) $\endgroup$ Mar 13, 2023 at 22:58
1
$\begingroup$

My view is from perspective of information.

A 1D vector of length $N$ has $N$ degrees of freedom. Every point is a free variable, and indeed the DFT is equivalently solving a system of $N$ equations. A vector hence has an $N$-dimensional extent of variability, instead of just 1D.

It's why audio is any challenge in machine learning: one approach is to reduce this variability (impose invariance) while retaining enough information to tell classes apart (preserve discriminability), which involves building a proper "latent encoding", which is fancy talk for "find other $M$ points with said properties".

A competing and equally important perspective is 1D: if we go double past Nyquist, we add no information, nor variability. This actually is what's most interesting in signal processing to me, it provides a tool to quantify the information capacity of finite data.

Which perspective is "right" depends on use. On derivatives, one can perfectly sample a derivative of a signal with no regard to vectors. Or, one can define the operation itself to be vector-based, as with $y_{i + 1} - y_i$. Latter has two interpretations: (1) approximating the continuous-time operation; (2) executing an exact operation, with the 1D sequence being ground truth. Former covers most use cases, and one can mathematically prove that "output" can be same for "get output by operating on discrete input" and "get output by sampling continuous-time operations". Latter is an infrequent perspective in SP, and it can justify classifying this as "beats" (... caveats attached).

And, it helps to contrast with fundamentally different structures: graphs. The key traits of "classical" signal discretizations is uniformity (adjacent points are equidistant over the information's domain of variation) and ordinality (it makes sense to say one point comes "after" another) - and graphs don't follow either. Indeed there is Graph Fourier transform, with graph convolutions.

when one thinks about the numerical gradient of this "vector", it is treated like a discretized 1D array, and the gradient becomes finite difference with one variable. It is no longer considered in R^n

Not really. The finite difference has $N - 1$ degrees of freedom - every point can vary freely and "react" to any point in the original sequence. Yet, why $N - 1$? Note, $d/dt (f(t) + C) = f'(t)$, for all $C$, and $C$ in no way contributes to $f'(t)$. One can show the same loss of information occurs with the finite difference (sum / DC bin), and that you cannot account for all $N$ input points (i.e. uniquely represent). And again important to mind both perspectives: you still get $N$ points by instead sampling $f'(t)$, yet info's still lost. May be of interest, DFT derivative property.

$\endgroup$
2
  • $\begingroup$ Interesting perspective. I think the problem with the vector label for a signal is the following: If we want add two 1D signals, that behaves like vector addition or multiply that by a scalar. When one thinks about the numerical gradient of this "vector", it is treated like a discretized 1D array, and the gradient becomes a finite difference with one variable. It is no longer in R^n. There are finite difference schemes where one can keep the same number of points, by using central difference for the all the points and for the boundary points (first/last), we define forward/backward difference. $\endgroup$
    – AChem
    Mar 14, 2023 at 16:30
  • $\begingroup$ "the gradient becomes a finite difference with one variable. It is no longer in R^n." I don't get what you mean. I addressed this at bottom, but maybe you're after something else. To put it differently, it's "one variable" in continuous time before and after differentiation, and the loss of one point doesn't fundamentally change the 1D vector. @AChem -- also this is great. $\endgroup$ Mar 15, 2023 at 11:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.