I am trying to develop a classifier for Indian Taals, for which I would be focusing only on the Tabla. Now once a extract a period of the Tala(a repetitive pattern), I want to develop some feature vectors for a Tala, which can be used to differentiate two Talas.

I am not sure what they should be. As of now, what I am planning to do is to find the beats of the Tala by beat tracking, or the onsets, and then take the median of the spectrogram values at the beat times to be my feature vector for a particular tala. Also, there is the concept of Bayan/Dayan in Tabla which have different pitches. I am not very sure how to use this information into my feature vector.

I would also want to include the time(temporal characteristics) into my feature.

Any help would be appreciated.


Very first thing to do: there is literature describing systems doing exactly what you want to do. Look for it!

Your suggested approach - representing the sequence as a feature vector of dimension $n$ where $n$ is the number of onsets or beats - has many flaws:

  • What if you want to classify sequences which are longer/shorter than your training samples?
  • What if the onset detector skips an event or detects a spurious event, and the entire vector is shifted left or right?
  • What about timing information, which is essential in the perception of rhythm!
  • A single real number computed from the spectrogram is unlikely to represent well the musicological dimension that matters to your problem (which "stroke" or "bol" has been played on the tabla).

Strings/sequence matching methods are more likely to be what you need. What matters to your classification problem is "what follows what" rather than "what happens at the $n$-th position" - because the concept of "$n$-th" position is not very robust, and also very relative.

Here's how I would approach the problem:

  1. Detect onsets on your audio signal, and for each onset, compute a vector of timbral features (MFCCs will be fine). At this point you have a sequence of onset times and a sequence of feature vectors.
  2. On your entire training database, train a vector-quantizer with a small-ish codebook (say 16 or 32 vectors) on the feature vectors. There's a good chance that it will map well to the tabla bols ('dha', 'ti', 're', 'tin'...) - each codeword will correspond to a distinct tabla stroke.
  3. Quantize your data using this vector quantizer. Your sequences will now be represented as a list of onset times and codeword id [(0.0, codeword_10), (0.51, codeword_7), (1.02, codeword_7), (1.27, codeword_16), (1.52, codeword_17), (2.03, codeword_10)...] etc.
  4. Use a classic tactus/tatum detection algorithm (they use inter-onset times as their input) to recover the temporal structure of the rhythm. This will allow you to group events into matras. The example above will become: codeword_10, codeword_7, codeword_7, codeword_16|codeword_17, codeword_10 (note the grouping - the fourth and fifth events are played in sequence within the same matra).
  5. Use any string/sequence modelling technique on the representation above:
    • Transforming the sequence obtained at step 4 into a sparse vector of n-gram counts is the most flexible method, since it allows you to use any text machine learning technique. The downside is that it needs lots of training data and works well on rather long sequences.
    • You can also use HMMs. Collect a small corpus of sequences in various taal, train a HMM on them for each taal; use the HMM to compute the likelihood of each of the taal models given the observed sequence.
    • Another option is to use an edit distance on the sequence obtained at step 4, in conjunction with nearest-neighbour classification. Compute the edit distance between the sequence to classify, and sequences representative of each of the different taals. Pick the taal with the smallest edit distance.

Note that if you have time to invest in manual annotation, steps 2 and 3 can be replaced by a supervised classifier: annotate a database of table sounds with the name or 'bol' of each stroke ('dha', 'dhin', 'ti', 're'...). Train a classifier from audio features (say MFCC) into bols. Use this classifier at step 3. You will now be able to work with a very meaningful representation of your sequences.

A final note: There's a good chance that step 5 can be done with a handful of rules, rather than through machine learning. I know the entire purpose of your project is to do tala recognition with machine-learning; but what is the point if a bit of musicological knowledge can do the same job? What machine learning can do is transcribe the audio signal into a symbolic representation (a sequence of bols) - but further processing of this representation to extract meaningful musicological properties/categories does not necessarily require machine learning techniques...

  • $\begingroup$ 1,2,3 are very clear to me. I could not understand the 4th step at all. I did not get the "group" and the "matra" thingy. From the definition here, a matra is a beat. So what does it mean here? Why couldn't I simply ignore step 4 and use the codewords from step 3? Could you please elaborate? $\endgroup$ Oct 22 '14 at 8:10
  • 1
    $\begingroup$ The goal is to group a sequence of strokes performed rapidly within the same 'beat' (matra) into a single entity, to distinguish them from a sequence of strokes performed slowly - at the rate of one stroke per beat. Let's say that you replace a "Dha Tin" by a "TiRe KiTe". Is it more correct to say that the length of the sequence has increased by 2 and that there are 4 changes; or that the length of the sequence is the same and that the Dha has been replaced by a TiRe and the Tin by a KiTe? $\endgroup$ Oct 22 '14 at 8:21
  • 1
    $\begingroup$ 1. If you decide to use supervised classification, each class will be a bol, yes. Any classification method (gaussian mixture models, support vector machines, neural networks) can be used - whatever works best! 2. I am not aware of any open-source implementation. $\endgroup$ Oct 22 '14 at 8:39
  • 1
    $\begingroup$ Since "TiRe" or "KiTe", consists of two strokes, you have to break these down into two events - so your classifier will include (among others), a class for "Ti/Te" (they almost sound the same), a class for "Re", a class for "Ki". Regarding MFCC, you can try using a short segment (say 50ms or 100ms) starting from the onset, or using the entire stretch of signal from one onset to the next - try both and use whatever performs best. As for the MFCC code, it shouldn't be hard to find - or you could try rolling your own just for the sake of learning new things... $\endgroup$ Oct 22 '14 at 9:10
  • 1
    $\begingroup$ No. MFCCs are calculated for a (small) signal window. Just increase the size of the window so that it covers the segment of audio you want to consider. $\endgroup$ Oct 22 '14 at 9:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.