# Tensor in signal processing

I saw a lot of use of tensor in signal processing. What is the intuition behind it? Is it simply a common representation for audio (1D), image (2D) and video (3D) signals?

This has little to do with intuition. Tensors are rigorously defined mathematical objects, and in general simple arrays don't qualify as tensors. Specifically, signals are not tensors.

In the language of mathematics, and there the field of differential geometry, a tensor is a linear function with several arguments. It is therefore also called a multi-linear form. It is used to describe properties of manifolds at a single point.

A point on a (smooth) manifold comes with a tangent space that contains all possible "direction" at that point, where a direction is really just a tangent vector at that point. This tangent space at a single point is a vector space and it has the same dimension as the manifold.

The tangent space has a natural so called dual space, which contains the linear functions that map tangent space vectors to real numbers. This is the cotangent space, and it's also a vector space of the dimension of the manifold.

A tensor is now (roughly) a linear function that maps any number of dimensions between the tangent space, the cotangent space and the real numbers.

Tensors typically don't come as singles but in so called bundles, which are the disjoint union of the tensors of all points of the manifold. For example if you have a vector at every point of the manifold then the whole structure is called a vector bundle. Similarly, tensor bundles associate a tensor with every point on the manifold, usually in a smooth way.

In signal processing you can encounter tensors, but usually they are called differently. For example in video processing the image is your manifold (or rather, a function on a 2-dimensional manifold giving the brightness or color for each point) and the velocity field that describes the local motion is a rank 1 tensor field (or bundle) on that manifold.

Tensors also show up a lot in volume data processing, where tensors can describe local properties of the volume data. For example if you have a doppler ultrasonic tomography image, then the velocity data in each voxel is a tensor field as would be mechanical stress in tomographic material analysis.

For a 1-dimensional signals tensors often come in the form of derivative operators. For example if you have a signal $s(t)$, then the operator $\frac{\partial}{\partial t}$ is a tensor field of rank 1, or just a vector field. More generally, if you multiply that operator with a smooth function $g(t)$, then all possible vector fields on the manifold the signal lives on are of the form $g(t)\frac{\partial}{\partial t}$

To sum up, tensors are a description of differential properties of manifolds. If you want to understand them, you'll have to study differential geometry.

• good answer, can you elaborate on the mapping of the tangent space vectors to real numbers? what does this real valued signal represent? – geometrikal Dec 20 '13 at 4:02
• @geometrikal: You can imagine this map as a generalization of the directional derivative of a function on the manifold and the construction of the gradient of that function. The directional derivative of a scalar function F at a point p will result in a real (or maybe complex) number that depends on the direction represented by a vector field at that same point p. Instead of evaluating the derivative of the function every time you change that direction v, you can also introduce the gradient of F at p and simply evaluate the scalar product <grad F,v>. – Jazzmaniac Dec 20 '13 at 11:18
• Modern differential geometry calls the linear function <grad F, * > a covector that live in the cotangent space at p. This abstraction allows you to remove the need of having a scalar product defined, because whatever linear product is used is already contained in the cotangent vector. These covectors are also called "differentials" and usually written dF. – Jazzmaniac Dec 20 '13 at 11:20

I think the question is not related to the tensor fields. There is a growing area in signal processing, namely tensor factorizations. Here tensor is simply a multiway array - generalization of the matrix. So, yes, a video is a tensor with dimensions $m\times n \times T$ here $T$ is a frame count.

Tensor related methods are not restricted to the video processing because in many data analysis problems, data comes in tensor form, for instance think of a tensor encodes the relations between $user \times user \times product$.

You can see an audio processing application of the tensor factorization from here.

• You're right that tensor product and tensor factorizations are important in signal processing. But contrary to common belief the objects they work on or result in are not automatically tensors. In fact, the typical tensor product works on ordinary vector spaces. So your conclusion that a video is a tensor is not really correct. At least from a mathematical stand point. – Jazzmaniac Dec 20 '13 at 11:11
• Yes, true, in this context tensor is just a name, from a rigorous point of view a video is not a tensor of course. But in this area, terminology goes like this in practice. – Deniz Dec 20 '13 at 15:21