2
$\begingroup$

I have an audio signal that contains, at various time offsets, various other short audio signals, often repeated, in addition to other audio content. Here is an example of how that could look like:

   0  1  2  3  4  5  6  7  8  9
A: x        x              x
B: x                       x
C:       x  x              

This terrible representation shows that A and B appear at 0 seconds, C appears at 2 seconds, A and C appear at 3 seconds and so on. These signals will additionally be scaled at every occurrence (their amplitude could be different at each occurrence) and possibly have an amplitude envelope applied to them.

Is there an algorithm to recover that takes all the position of the occurrences and recovers (to the best of its ability) A, B, and C?

To be more explicit in the task I have to solve, I have an old electronic music track I produced, but I lost the project file. Like most electronic music, it's a mix of synthesized (pseudoperiodic) sounds, electronic drums hits (called, confusingly, samples), vocals, effects and maybe some acoustic instrument. All those signals get summed together and processed in a "mastering chain": the thing relevant to this problem is that dynamic processing is applied to increase the average loudness, and saturation (in the case of this track, just soft clipping) is added to squeeze another bit of volume.

I need to change one of the drum hits. I don't need to recover it perfectly, because even if a bit of the old drum hit and a bit of noise is still there, the new drum hit that I'm going to replace it with will cover it.

$\endgroup$
7
  • $\begingroup$ PS: I don't actually know much about signal processing, but I've stumbled on it after having this problem and I'm starting to study it. Fire away whatever math wizardry is needed and I'll be happy to study more! Also feel free to tell me if this is dumb and impossible :P $\endgroup$
    – user29639
    Jul 9, 2017 at 16:00
  • $\begingroup$ that representation isn't terrible at all :) What does the "scale" within "will […] be scaled at every occurence" mean? $\endgroup$ Jul 9, 2017 at 16:03
  • $\begingroup$ It means that at 0 seconds it could be A, while at 3 seconds it could be 0.8 * A. How do I write that more properly? $\endgroup$
    – user29639
    Jul 9, 2017 at 16:07
  • $\begingroup$ "it's amplitude could be different at each occurrence", but reading it again, I might be the only one who would not understand it like that the first time :) $\endgroup$ Jul 9, 2017 at 16:12
  • $\begingroup$ You can perhaps attack this using non-negative matrix factorization. Please see this question: dsp.stackexchange.com/questions/35878/… and script here: nbviewer.jupyter.org/gist/ingle/… $\endgroup$
    – Atul Ingle
    Jul 9, 2017 at 16:54

1 Answer 1

3
$\begingroup$

This is a source separation problem, which, in general is quite difficult to solve. One way to attack it is to use non-negative matrix factorization (NMF) with short-time Fourier analysis. Please see this question and a sample script here.

The overarching idea is to try to isolate your component sounds A, B, C,... using a time-frequency decomposition (or, perhaps, some other kind of subspace analysis using a different transform, like wavelet transform).

If the components don't always all play at exactly the same time, or have different onsets, you can hope to separate them temporally. If they don't all have identical frequency content, you have some hope of separating them in frequency. Or, a combination of both, which is what NMF of the STFT achieves.

$\endgroup$
2
  • $\begingroup$ I will wait for the script to finish, check other answers if there are any, then accept this one. Thanks! $\endgroup$
    – user29639
    Jul 9, 2017 at 19:13
  • $\begingroup$ I'm having some problems with the high frequency components (and by extension the snare), but I have a couple of ideas that I want to try, so for the time being i'm accepting this. $\endgroup$
    – user29639
    Jul 12, 2017 at 10:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.