# Recognizing math functions within songs

I'm new to DSP, and just discovered this StackExchange, so apologies if this isn't the right place to post this question.

Is there a resource that describes genres in a more mathematical terms? For example, if I've performed an FFT on the signal on this section of the song (2:09 if the link doesn't start there), is there any way of me being able to detect that this section has that rough sort of sound? Do sounds like this follow some mathematical function with which I can compare? http://www.youtube.com/watch?v=SFu2DfPDGeU&feature=player_detailpage#t=130s (link starts playing sound straight away)

Is the only way to use supervised learning techniques, or is there a different approach (which preferably doesn't require supervision)?

• You could use an FFT to detect a drum beat vs a flute, but not to detect genre. Very local stuff about sound, sure, but not the total file's musical character. – endolith Sep 21 '11 at 16:23
• Is it possible to recognise the "roughness" of the sound? Is this where MFCC's come into play? – XSL Sep 21 '11 at 21:31

I think the distinction you're looking for is more like empirical vs. theoretical (as opposed to supervised vs. unsupervised), but I could be wrong about that. In other words, the ideal thing would be to have a theoretical definition of various genres, rather than just a bunch of opaque data which can be used to classify a song [without any real understanding].

However, for general genre classification, you're probably stuck at least with training from examples, even if just to create the definitions of genres in the first place. With respect to your example, consider how frequently people will argue [on YouTube] over whether a given track is really dubstep (e.g. any track that's more dubby and less wobbly, even though the genre started out without any real wobble). People define genres over time through examples, so it's reasonable to expect that algorithms which replicate that behavior would also require some examples. The way people describe genres is almost like a feature vector anyway--they ask a list of questions about the song (e.g. Is it more breaky or wobbly? Does it have a lot of sub bass? How long is it? What's the tempo? Is there a vocal? etc.).

Of course, you may be able to choose a list of features that also provide an intuitive understanding of the genre. A feature like "Dynamic Range" is something that a person can also detect by ear, but something like "Time Domain Zero Crossings" wouldn't be very intuitive--even if it works well for classification. The following paper has quite a few features that might be interesting to you:

George Tzanetakis, Perry R. Cook: Musical genre classification of audio signals. IEEE Transactions on Speech and Audio Processing 10(5): 293-302 (2002) link.

For measuring roughness, psychoacoustic roughness would be a good place to start, but it might not be sufficient to distinguish between dubstep leads and electro leads, for example. For finer-grained distinctions, one thing to look into is timbre recognition. The following thesis has a decent survey of techniques:

T. H. Park, “Towards automatic musical instrument timbre recognition,” Ph.D. dissertation, Princeton University, NJ, 2004. link.

There's also a model related to perceptual roughness in Timbre, Tuning, Spectrum and Scale which is used for constructing custom scales for arbitrary timbres. The idea is that harmonics which are very close together produce beat frequencies which are perceived as dissonance. Paraphrasing from Appendix F and E,

When $$F$$ is a spectrum with partials at frequencies $$f_1,f_2,...,f_n$$, the intrinsic dissonance [assuming unit amplitudes] is

$$D_F = 1/2 \space \sum_{i=1}^{n}{} \space \sum_{j=1}^{n}{\space d\left({|f_i - f_j| \over{\min(f_i,f_j)}} \right) }$$

where $$d(x) = e^{-3.5 x} - e^{-5.75 x}$$

is a model of the Plomp-Levelt Curve.

It's used for measuring how pleasing a given chord is with respect to a timbre (by minimizing the dissonance). I don't know if either roughness of the psychoacoustic variety, or intrinsic dissonance would be very fruitful for your purposes on their own, but they may be useful in combination with other metrics.

You'll probably have more luck classifying timbres mathematically than genres. For example, strings have even and odd harmonics, but a clarinet has only odd harmonics (cf. Sawtooth wave, Square wave). Dubstep wobble tends to be done with LFO-driven filters (low pass and/or formant filters), so something like Spectral Flux (see [Tzanetakis], above) might be a good starting point as a feature. However, I doubt anyone has studied mathematical classification of wobble yet ;)

• Excellent response from datageist. I would also suggest isophonics.net/QMVampPlugins if you are looking for more information and a dev platform with source code – Dan Barry Sep 22 '11 at 10:36
• @Dan That looks awesome, thanks for that link. – datageist Sep 22 '11 at 10:39
• I've found a niche subject to study! :D Thanks for the brilliant answer and links. It's pointed me in a direction rather than aimlessly using Google. – XSL Sep 22 '11 at 14:41
• Glad to help :) – datageist Sep 22 '11 at 14:53