Playing a piece of music audio at a slower speed would lower its pitch (frequency). Is there a tool and theory to slow down the song playing while keep the frequency the same? I suppose one can do windowed Fourier transform or wavelet transform. It seems one have to either pre-select the window size or dynamically choose the wavelet basis. Is there any specific and detailed theory and application for doing that?
1
-
1$\begingroup$ In addition to the answer provided below, please see this link for more information. $\endgroup$– A_ACommented Jul 1, 2016 at 9:29
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
3
Yeah some of us can do it, you can speed up or slow down without affect the pitch, some guys call this applications of Time Stretch, there different ways to do it, you can do in frequency domain or time domain, you will need choose what is best for you, you will find some advantages and disadvantages of each.
Time Domain:
In Time Domain you can try some techniques like:
- TDHS(Time domain Harmonic Sacaling)
- SOLA (Synchronous Overlap Add)
- PSOLA(Pitch Synchronous Overlap Add)
- WSOLA(Waveform Similarity Overlap Add)
Pros: Is fast, some algorithms are easy to understand, good quality in monophonic sounds.
Cons: Generally you'll need a very nice pitch track to splice in the right position, it is hard to do :-(, so if your pitch track fails or not work in Poliphonic sounds this algorithms will give a lot of glitches/artifacts in the output sound.
Frequency Domain:
All time stretch that i know in frequency domain are based in phase vocoder techniques.
Pros: Will work in polyphonic or monophonic sounds.
Cons: Can be painful understand all the math, implementation is a bit hard, is not so fast like time domain codes, for voice i prefer time domain results, some tricks to improve the result of the standard phase vocoder are not shared.
I can say that the window and the hop size are one of the key for the phase vocoder quality, generally we choose 4x overlap to resynthesis, one hann window of size 4096 is enough for my ears (of course if u have processing power for this sizes), the standard phase vocoder can add some reverberant effetcts, to try avoid this kind of problems you maybe need lock the phase.
For datailed take a look in the Miller Puckette and Portnoff paper
-
$\begingroup$ Thank you for your answer. What is the problem with the most naive of the approaches: expand the time signal on the whole as a function on the whole time interval without windowing into Fourier series and multiply all frequencies by a constant. I understand any local error in time domain would affect all Fourier coefficients. Aside from that, what are the pitfall of this naive non-localized approach? $\endgroup$– HansCommented Jul 1, 2016 at 17:56
-
$\begingroup$ I've never tried anything like that, it may work, the obvious problem is that this can be quite costly, It is certainly not an efficient way, imagine that you have a piece of audio (1 minute) sampled at 44100Hz, now to do what you are proposing you will have to apply fourier at
44100 * 60 = 2646000points at once and process, so forget to any attempt at real-time processing such this, $\endgroup$ Commented Jul 2, 2016 at 0:13 -
1$\begingroup$ I do not think what I suggested before would work in the pure mathematical sense (disregarding the cost and error sensitivity). $\endgroup$– HansCommented Jul 8, 2016 at 20:26
$\begingroup$
$\endgroup$
2
The tool/theory you describe is really a large area of research in music technology, broadly called audio time-scale modification. A large component of this field is how you might prevent audible changes to frequency following time stretching. This can be approached with both frequency- and time-domain methods, depending on the constraints or goals of your application. The wikipedia entry for Audio time-scale / pitch modification is a good starting point.
If you're keen to pursue an approach using frequency/wavelet basis, your window-size and choice of basis will affect how well you're able to localise the signal. To use the STFT as an example, a long window will perform well for stationary sinusoids but destroy your transients. A shorter window will provide a preferable transient response at the cost of frequency-domain localisation. The performance of other wavelet bases will depend on the nature of the projection of your signal onto the basis.
-
$\begingroup$ Thank you very much for the answer. Do you have any reference on the application of wavelet to this problem? $\endgroup$– HansCommented Jul 1, 2016 at 17:49
-
$\begingroup$ Please see my comment below ederwander's answer as well. Thanks. $\endgroup$– HansCommented Jul 1, 2016 at 18:12
$\begingroup$
$\endgroup$
6
Below is a link to a simple and valuable tutorial function in C++ (smbPitchShift.cpp) by Stephan M. Bernsee, which can slow-down or speed-up music without changing its pitch.
He has released this code under the The Wide Open License (WOL). Within my application, I was able to adapt his function to slow-down music in real-time -- that is while playing a mp3 file and additionally doing pitch detection upon that mp3 signal at the same time.
I have also included a link to Bernsee's website which contains his detailed descriptions on the Time-Stretching and Pitch-Shifting of audio signals, such as music.
answered Jul 5, 2016 at 21:34
-
$\begingroup$ The original code does not apply
time stretch, the original code applyPitch Shift, in this case to apply time scale modification you will need to combinePitch Shifit + Resample (interpolation), The Bernsee's code works well using a window of size4096you will be able to pitch shift one octave (above or below), it means that accordingly you will only be able to make time scale with a good quality using factors between 2.0x-0.5x, an phase vocoder well built can achieve better results using the same window size, and you will be able to extrapolate these factors with better quality $\endgroup$ Commented Jul 6, 2016 at 11:46 -
$\begingroup$ Oops, now remembering that I had to apply Re-Sampling to complete the time-stretch, so that the original pitch was not altered. Looks like Bernsee has created some changes to his code since the version linked at GitHub. His newer code for is available for download from his website -- it may increase the range of shifting from his original specification. I tweaked his original code so that I could pitch shift up by 8x. $\endgroup$ Commented Jul 6, 2016 at 17:24
-
$\begingroup$ there are no differences between your code and the Bernsee page, the main math is still the same, The strong difference that I can see is the window size =
8192in the codes from download page, so do you have to do 4x more points to processing, I come back to say that with half window size8192/2=4096you can do the same using some secrets of phase vocoder, the point here is that you can keep the quality with much less processing. $\endgroup$ Commented Jul 6, 2016 at 17:54 -
$\begingroup$ Though I provided the GitHub link to BatPhone, it is NOT my code. I just pulled it from an Internet search to give viewing of smbPitchShift(). My code is much modified from Bernsee’s version, and resided in this file: github.com/CreativeDetectors/PitchScope_Player/blob/master/Src/… $\endgroup$ Commented Jul 6, 2016 at 18:22
-
$\begingroup$ Yeah now i can see, it is really an resample, you may be interested to see / hear my phase vocoder in action. $\endgroup$ Commented Jul 6, 2016 at 18:48