How is a discretized 1D signal be considered a 'vector'?

Question

This question is perhaps related to the semantics or jargon of signal processing. I have checked plenty of advanced books/monographs on multivariable calculus and signal processing but could find an explicit answer.

Suppose we have a simple function $y$=$\sin(x)$. In short, if we have sampled a sine wave as a function of time, we will have discrete values of the sine wave corresponding to a uniform grid of time. We can list all the $y$ values in a column, and call it a signal vector.

Now how can one interpret this as a vector? Is this a point in an $n$-dimensional $\mathbb{R^n}$, which is perhaps not true for 1D discretizes signals, but certain book on Fast Fourier Transform does interpret it that way. For example, Mathematics of the Discrete Fourier Transform writes in Chapter 5,

5.2 Signals as Vectors For the DFT, all signals and spectra are length $N$. A length $N$ sequence $x$ can be denoted by $x(n), n=0,1,2, > \ldots N-1$, where $x(n)$ may be real $\left(x \in \mathbf{R}^N\right)$ or complex $\left(x \in \mathbf{C}^N\right)$. We now wish to regard $x$ as a vector ${ }^1 \underline{x}$ in an $N$ dimensional vector space. That is, each sample $x(n)$ is regarded as a coordinate in that space. A vector $\underline{x}$ is mathematically a single point in $N$-space represented by a list of coordinates $\left(x_0, x_1, x_2, \ldots, x_{N-1}\right)$ called an $N$-tuple. (The notation $x_n$ means the same thing as $x(n)$.) It can be interpreted geometrically as an arrow in $N$-space from the origin $\underline{0} \triangleq$ $(0,0, \ldots, 0)$ to the point $\underline{x} \triangleq\left(x_0, x_1, x_2, \ldots, x_{N-1}\right)$. We define the following as equivalent: $$ x \triangleq \underline{x} \triangleq x(\cdot) \triangleq\left(x_0, x_1, \ldots, x_{N-1}\right) \triangleq\left[x_0, x_1, \ldots, x_{N-1}\right] \triangleq\left[x_0 x_1 \cdots x_{N-1}\right] $$

This is exactly the same notion of vectors in vector calculus books

This notion is fine but this conceptual problem starts when one starts to think about the numerical derivative of this sine function "vector". The 1D dimensional gradient is easy to visualize since it will just be $y_{i+1} - y_i$, but if this vector is a point in $n$-dimensional space, how do we think about its derivative? I feel some important notion is missing here.

In short, how can one distinguish that this given "vector" is a discretize time based signal of one variable such as time or this signal is a vector in N-dimensional space $\mathbb{R^N}$

I have not formally studied DSP, however this interest is due to signal processing in chemical problems.

Answer 1

This is exactly the same notion of vectors in vector calculus books

Yes, and in fact, the DFT maps the $\mathbb R^N$ (or $\mathbb C^N$) time-domain space to the $\mathbb C^N$ space -- and you can express that mapping as a matrix multiply.

Note, too, that any 1D vector can be taken as a function of one variable. Just let $x(k) = x_k$, and there you are.

Perhaps your difficulty stems from the fact that in signal processing we usually want to think of a signal as something evolving in time (or space), but for analysis we still often want to use the tools of Linear algebra or multivariate calculus. When we want to do the latter, we declare our signal a vector so that we can use those tools on it.

The 1D dimensional gradient is easy to visualize since it will just be $y_{i+1}−y_i$, but if this vector is a point in n-dimensional space, how do we think about its derivative?

I think your thinking may be a bit too concrete here. But you could certainly consider the derivative of $\mathbf y$ to be a mapping either $\mathbb R^N \to \mathbb R^{N-1}$ (if you want to disallow the $n^{th}$ value of the first difference) or $\mathbb R^N \to \mathbb R^N$ (if you wanted to allow it somehow, possibly by defining your first derivative as $y_{i + 1\bmod N\ -\ y_i}$).

I.e., if you define the first difference of $\mathbf y$ as $\Delta \mathbf y$, then there's a matrix $\mathbf A$ that implements $\Delta \mathbf y = \mathbf A \mathbf y$. $\mathbf A$ can be either $N \times N-1$ or $N \times N$, depending on whether you want a wrapping on non-wrapping first difference.

I will say this -- some of this signal processing stuff can be quite counter-intuitive until you've worked through a great number of examples. I remember having to go on some very long walks and very long bicycle rides to really get my head wrapped around it.

Answer 2

My view is from perspective of information.

A 1D vector of length $N$ has $N$ degrees of freedom. Every point is a free variable, and indeed the DFT is equivalently solving a system of $N$ equations. A vector hence has an $N$-dimensional extent of variability, instead of just 1D.

It's why audio is any challenge in machine learning: one approach is to reduce this variability (impose invariance) while retaining enough information to tell classes apart (preserve discriminability), which involves building a proper "latent encoding", which is fancy talk for "find other $M$ points with said properties".

A competing and equally important perspective is 1D: if we go double past Nyquist, we add no information, nor variability. This actually is what's most interesting in signal processing to me, it provides a tool to quantify the information capacity of finite data.

Which perspective is "right" depends on use. On derivatives, one can perfectly sample a derivative of a signal with no regard to vectors. Or, one can define the operation itself to be vector-based, as with $y_{i + 1} - y_i$. Latter has two interpretations: (1) approximating the continuous-time operation; (2) executing an exact operation, with the 1D sequence being ground truth. Former covers most use cases, and one can mathematically prove that "output" can be same for "get output by operating on discrete input" and "get output by sampling continuous-time operations". Latter is an infrequent perspective in SP, and it can justify classifying this as "beats" (... caveats attached).

And, it helps to contrast with fundamentally different structures: graphs. The key traits of "classical" signal discretizations is uniformity (adjacent points are equidistant over the information's domain of variation) and ordinality (it makes sense to say one point comes "after" another) - and graphs don't follow either. Indeed there is Graph Fourier transform, with graph convolutions.

when one thinks about the numerical gradient of this "vector", it is treated like a discretized 1D array, and the gradient becomes finite difference with one variable. It is no longer considered in R^n

Not really. The finite difference has $N - 1$ degrees of freedom - every point can vary freely and "react" to any point in the original sequence. Yet, why $N - 1$? Note, $d/dt (f(t) + C) = f'(t)$, for all $C$, and $C$ in no way contributes to $f'(t)$. One can show the same loss of information occurs with the finite difference (sum / DC bin), and that you cannot account for all $N$ input points (i.e. uniquely represent). And again important to mind both perspectives: you still get $N$ points by instead sampling $f'(t)$, yet info's still lost. May be of interest, DFT derivative property.