# Synthetic sounds to describe motion in 3d or 100d

Watch a butterfly flitting about, or an optimizer chugging along in 3d or 100d -- a sequence of points $X_1 \ X_2 \ X_3\ \ldots$

How could one generate synthetic sounds that convey

• moving fast or slowly
• moving steadily in one direction, or oscillating ?

The question has two parts (which should perhaps be asked separately):

1. how to filter a high-dimensional path to a 1d signal that conveys the kind of motion
2. how to convert that to pleasant sounds.

The goal would be to listen to sounds made by a running optimizer as it moves along in 3d or 100d, and glean some info on how it's moving.

• First question is nonsensical, because no projection in this world could map map something from $\mathbb K^{100}$ invertibly to $\mathbb K^1$. Think about it: there's infinitely many 3D objects that have the same shadow. In general, these objects could even be moving without changing the shadow. So there's no way to represent a 100D-movement in 1D, unless the movement was originally along a single dimension, just in a 100D space... – Marcus Müller Jun 11 '16 at 14:27
• You could try to track the speed of motion (assuming it is not changing direction), but you would immediately end up doing differential geometry... Um... how do I put this: That might be a bit much math to really simplify your problem. – Marcus Müller Jun 11 '16 at 14:29
• @Marcus Müller, of course it can't be invertible; but do you ever hear noises (screech, bumpy road ...) from your car ? SVD / PCA map to low-d, but I don't see how to do piecewise PCA. – denis Jun 11 '16 at 15:06
• Well, that's really a question for the nature of your data, because obviously, the PCA component you chose will depend on which piece of the data you consider, so the selection of the pieces is not neutral to the PCA, and only strong statements on how your data behaves will help you here; that's why I said this might quickly become hard math: finding a surface on which your measurement points lie and mapping some characteristic of that surface (along a path, maybe?) to 1D will only make sense when you consider the "roughness" or "curvature" of the data itself, which again, is a hard problem – Marcus Müller Jun 11 '16 at 15:30
• But you state your points are strictly ordered - "a sequence of points". Well, in that case, you can either define "speed" as "euclidean distance (or any other metric) per in between two consecutive points", or you can define speed as the first derivate of the interpolating polynomial (or any other interpolator, but polynomials are easy to derive), but then you'd have a $N-1$-degree polynomial over $\mathbb K^100$, with $N$ being the number of points, which might, or might not, be handy. Then map speed to some kind of tonal scale, or loudness. – Marcus Müller Jun 11 '16 at 15:35

You say your points are strictly ordered:

a sequence of points $X_1 \ X_2 \ X_3\ \ldots\ X_N$

Well, in that case, you can

1. either define "speed" as "euclidean distance (or any other metric) per in between two consecutive points": $$s_n = ||X_{n+1} - X_n||,\quad n=\{1,\ldots,N-1\}$$ and interpolate a steady function $s(t)$ from that sequence, or
2. you can define speed as the first derivate of an interpolation function through all $X_i$. However, finding an interpolator for $N$ points in 100 dimensions isn't easy, and also, its utility would be pretty questionable, unless you have some goal in mind that clearly says why a specific interpolator is appropriate for your problem.

After you have a scalar value along a single axis, well, map to tone or loudness.

I really doubt the "illustrating" effect is overly great here: the choice of your interpolation will probably have the most important effect on what listeners will take as an impression from this, and that choice is not backed by the data itself – it's a model choice you make. Which would be ok, if you know something about the data (i.e. all the individual dimensions just contain harmonic oscillations would suggest classical low pass filtering would be an appropriate method to interpolate "speed" points), but if you don't and want to use this "audiolisation" to understand your data better, you'd end up fooling yourself. You will listen to your own assumptions, not the data.

However, you mentioned PCA, which indicates the points aren't actually to be understood sequentially! That of course makes things a lot harder. You'd essentially be looking for a $100-1$ hypersurface that describes how your points are mainly distributed in your $100$ dimensional space, and then look for the main gradient of that – and that leads to all the nice problems that you get with higher dimensional analysis (lack of "one" derivate of function, mapping values from something that isn't a vector space to a vector space, which introduces the need for maps on manifolds... meh).

The idea of generating a random sequence, modifying it and producing "music" with it is not new. Swarms have in fact been suggested for music composition and "Evolutionary Music" is a very actively researched area (e.g. see this link).

So, there might be some background already that you might want to consult before embarking to your own experiments. Now, as far as your questions are concerned:

how to filter a high-dimensional path to a 1d signal that conveys the kind of motion

I wonder if by filter, here, you actually mean, project. You can definitely use an $f(\Theta)$ to combine in an arbitrary way individual components of some set of vectors $\Theta$. In fact, you could even have different $f_n(\Theta)$ to represent different sources of sounds. (e.g. different instruments, or different evolving parameters of the same source). Of course, once a single vector has been produced (at the output of one of those $f$s above), then actually filtering it is just a case of applying some digital filter)

how to convert that to pleasant sounds.

The keyword here is pleasant. And of course ,subsequently we might ask "for whose ears?". Pleasant is subjective. You can achieve Pleasant by simply mapping a sequence to a music scale. In fact, you don't even need complex bee swarm models for this. Just produce numbers from different distributions and then map them on the notes of a scale and send them out to MIDI. That will give you something Pleasant. For example, consider the permisible tones in a C major scale: C D E F G A B. For a given sequence of random integers, do a modulo by 7 and then map them to MIDI messages at some tempo (e.g. eighths at 92 BPM). This will give you a pleasant sounding melody, rather than, what you get when you feed that same sequence of random numbers directly to your speakers (e.g. white noise). Now, to change the "feeling", keep the sequence the same but change the scale that these numbers map to to: C D Eb F G Ab Bb. That's a minor scale. Different intervals between the tones, different end-result.

And this in fact, is a better point of focus than Pleasant. That is, Harmony. And here is a piano to play music in a non-standard harmony, harmony that emerges by dividing the semitone. A different kind of Pleasant.

Hope this helps.