0
$\begingroup$

This tool (Paul's Extreme Sound Stretch) is able to produce extreme timestrech (50x, or even 100000x). Any song can become an interesting sound texture or ambient music (listen to this example).

The technique used is mentionned as:

"spectral smoothing" the sounds.

How does an extreme time stretch algorithm work?

(it is opensource, I have studied it a bit, but I haven't been able to see a big picture of the algorithm yet)

I thought about :

sound input ====>  STFT ===> duplicate the frames 50x ===> ISTFT ===> output

but I'm really not sure about this.

$\endgroup$
  • $\begingroup$ Your scheme would stay the same if you just left out the stft/istft steps. Block repetition in time domain gives exactly the same result. $\endgroup$ – Jazzmaniac Aug 29 '15 at 10:17
  • $\begingroup$ I can't hear your example $\endgroup$ – gmotree Aug 29 '15 at 11:53
  • $\begingroup$ Basj, are you well familiar with the phase vocoder, particularly what is described by Miller Puckette in this paper? i must disagree with other answers here that say to disregard phase. but what you must do (what Miller suggests) is, in your analysis frame, to identify and separate sinusoidal components and adjust the phase on all FFT bins for a particular component together as one phase adjustment. you can use a well-designed phase vocoder to slow any audio down to a crawl. $\endgroup$ – robert bristow-johnson Aug 29 '15 at 23:03
  • $\begingroup$ @robertbristow-johnson, see my comment at hotpaw's answer regarding why in the case of large stretching factors phase plays a different role. $\endgroup$ – Jazzmaniac Aug 30 '15 at 13:09
  • $\begingroup$ a good and robust algorithm should work the same in any case. whether the stretching factor is extreme or not. $\endgroup$ – robert bristow-johnson Aug 30 '15 at 21:23
2
$\begingroup$

Here is how the algorithm works:

enter image description here

So the steps are:

Input -> apply some window (hann in the frame) -> FFT -> Get amplitude > randomize the phase -> IFFT -> overlap and add -> out

Do it in a loop based in the strech Factor.

Paul wrote a Python code for all steps in this picture.

$\endgroup$
  • $\begingroup$ If you work in time domain, an TDHS ( Time Domain Harmonic spectrum) can be another way ... $\endgroup$ – ederwander Aug 29 '15 at 22:39
  • $\begingroup$ I think it's "scaling", not "spectrum". $\endgroup$ – Jazzmaniac Aug 30 '15 at 15:08
  • $\begingroup$ Ow thanks to remember me, is a time domain scaling $\endgroup$ – ederwander Aug 30 '15 at 19:50
1
$\begingroup$

At such large stretching factors, you can generously ignore phase information of the source signal and also assume the signal to be stationary.

What you can do is use a windowed short time FT to analyse a single frame, take the magnitude of the Fourier coefficients and smooth them to get an approximate spectral shape for the current frame. For synthesising a stretched signal you can just feed white noise through a convolution with the smoothed spectrum to get a stationary signal with the right spectrum.

The length of the noise signal for each frame determines the stretch factor, and fading between the resulting signals of adjacent blocks will give you the impression of an evolving sound.

Make sure that the noise is uncorrelated between frames, i.e. don't use a pre-calculated noise vector.

If you want to process stereo signals, it is probably best to first decompose the stereo signal into its mid and side channels, stretch those independently and convert back to left/right.

$\endgroup$
  • $\begingroup$ Thanks for your answer. Ok for taking the magnitude of Fourier coefs. But then, what do you mean exactly by smooth them and get an approx spectral shape for current frame? Can you give an example? Sorry to bother you, but could you describe your algorithm in pseudo code? I'll write a Python version and will post the result on Github then. ~ ~ Last thing: i didn't understand the noise + convolution thing. Why noise and what's the idea behind? $\endgroup$ – Basj Aug 29 '15 at 12:43
  • $\begingroup$ Basj, send me an email (i'm easy to find on the internet) and i'll send some MATLAB code about how to do this. sorry, i am still a Python luddite. $\endgroup$ – robert bristow-johnson Aug 30 '15 at 21:25
1
$\begingroup$

Another way to do extreme time stretching is to pay attention to FFT phase. Use the relative phase difference between two close adjacent analysis frames to do (a better than bin quantized) phase-vocoder frequency estimation of local peaks in a "smoothed spectrum", and then synthesize (continuously, not frame based) from those local peaks plus the addition of any non-peak noise floor.

$\endgroup$
  • $\begingroup$ this is the same that i know as locked phase vocoder to me ?? $\endgroup$ – ederwander Aug 29 '15 at 21:10
  • $\begingroup$ Thank @hotpaw2. Can you explain a bit in pseudo-code or block diagram like ederwander's answer, it would be great and easier to understand the idea... Thanks! $\endgroup$ – Basj Aug 29 '15 at 21:15
  • $\begingroup$ @hotpaw2, that works reasonably well for small stretch factors where phase coherence plays an important role. For very large stretch factors however, noise will turn into tonal features that are perceived as artifacts. That is why for extreme factors the assumption of spectral stationarity is useful to fill the noise (i.e. greater bandwidth) parts of the spectrum with actual noise. $\endgroup$ – Jazzmaniac Aug 30 '15 at 13:07
  • $\begingroup$ @Jazzmaniac it really makes sense, but i remember that when a wrote an TDHS code was possible stretch monophonic input to the hell, the same can be done using a locked phase vocoder, all depends on the effect you want to achieve,I think it is possible keep phase coherence and do extreme stretching without much perceptible artifacts. $\endgroup$ – ederwander Aug 30 '15 at 14:13
  • $\begingroup$ @ederwander, sure, it will work, but it will make noise become tonal. So if you want to generate soundscapes that evolve over time it doesn't make a huge difference. But keeping the noise noisy can actually help to create a more interesting picture and preserve the real harmonic structure. And disregarding phase of course makes the whole business a lot simpler. $\endgroup$ – Jazzmaniac Aug 30 '15 at 14:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.