# Isolate recurrent pattern vs. evolving background sound

Here is a sound example for what follows.

Let's assume we have a signal

$$s(t) = r(t) + e(t)$$

where:

• $r(t)$ is a signal which is recurrent with a given period, i.e. in my example $r(t) =$ the exact same snare sample, which is played every 1 second, starting at 00:00:00,500

• $e(t)$ is an evolving sound (in my example it's the synth pad soft sound)

I assume the signal is mono, and that no compression or other master effect has been applied.

Question:
How to recover $r(t)$ from the mix $s(t)$, by analyzing which "pattern" is recurrent in $s(t)$? (the period 1 second is considered as given)

In one word, how to isolate the snare sound from the mix?

I would like to do it by slicing the signal each 1 second:

and analyze what part of signal is recurrent (= the snare) and which is not (= the synth) among all slices ?

Is there a separation algorithm based on analyzing recurrent parts / non-recurrent parts? Using FFT or something else?

• So what is the output you want? The train of snare sounds AND the unmixed pad sound? Is there any reason why you must slice each 1 second? – ruoho ruotsi Nov 11 '15 at 0:29

You should try "Harmonic Percussive Source Separation" ... quickly testable on your WAVs using librosa's librosa.decompose.hpss(spectrogram):

Median-filtering harmonic percussive source separation (HPSS). Decomposes an input spectrogram S = H + P where H contains the harmonic components, and P contains the percussive components.

Here are some citations and papers:

Fitzgerald, Derry. "Harmonic/percussive separation using median filtering." (2010). (PDF)

Jeongsoo Park, Kyogu Lee, "Harmonic-Percussive Source Separation Using Harmonicity and Sparsity Constraints", Proceedings of International Society for Music Information Retrieval Conference (ISMIR), pp. 148-154, Malaga, Spain, 2015 (PDF)

Code example:

import librosa

y_harmonic, y_percussive = librosa.effects.hpss(y)
librosa.output.write_wav('output_harmonic.wav', y_harmonic, sr)
librosa.output.write_wav('output_percussive.wav', y_percussive, sr)

• okay, lemme whip up an example for you. I'll append it to my response when its ready :) – ruoho ruotsi Nov 18 '15 at 19:41
• Excellent! I'm really glad that worked out for you! I personally like it as a way to preprocess full-mixtures of sound before doing onset/beat-detection. Cheers! – ruoho ruotsi Nov 21 '15 at 3:05

So, assuming the linear superposition you suggest, the sound you want to exctract lives in a periodic subspace of your signal space. This subspace is uniquely characterised by the exact spacing between the sound instances, which I assume you know in the following. If you don't know it, you can find it from calculating the autocorrelation sequence of your composed signal.

Without further knowledge, the only linear method for finding your sample is to project the composed signal onto the periodic subspace it lives in. There are two obvious ways you can do this.

The first is taking the Fourier series of the composed signal and nulling all coefficients that do not belong to an integer multiple of the frequency corresponding to the period length of the repetition. Summing the modified series will give you the desired signal.

The second is to just accurately cut the sample into segments that align perfectly with respect to the repetition period and just sum all those segments up into a single sample.

The result of these two operations will give you a representation of the composed signal the same periodic subspace. However, since the subspace is relatively large compared to the total signal, there will be components of the composed signal that are contained in this subspace by mere chance. So the separation will be far from perfect. You can improve the separation by taking more repetitions, make the full signal space much larger than the periodic subspace.

If you need a better method you won't get around characterising the nature of the signal you want to extract. That's where perception models, source models, statistical signal models, etc come in. I cannot really go into this in this answer. It's a topic that fills many books and is in general very difficult.

This can be approached as a general blind source separation problem with an excitation-filter model for both the snare and the synthesizer. For the snare the excitation would be an impulse train, and the filter would have the snare sample as its impulse response. You can try if Flexible Audio Source Separation Toolbox (FASST) finds good enough models to allow separation. FASST is described in these articles:

Y. Salaün, E. Vincent, N. Bertin, N. Souviraà-Labastie, X. Jaureguiberry, D. T. Tran, and F. Bimbot, The Flexible Audio Source Separation Toolbox Version 2.0, in Show & Tell, IEEE International Conference on Acoustics, Speech and Signal Processing, 2014.

A. Ozerov, E. Vincent, and F. Bimbot, A general flexible framework for the handling of prior information in audio source separation, IEEE Transactions on Audio, Speech and Signal Processing 20(4), pp. 1118-1133 (2012).