Programming a Vocoder

Question

I'm want to program my own vocoder synthesizer like "Songify" but I can't find a simple of vocoder algorithm. Maybe you can explain or tell where to look for information on how vocoder works.

Answer 1

I fear that all answers here are irrelevant to the question. What is called a vocoder in the music production world has little to do with the phase vocoder used in signal processing. To make matters worse the Songify app referenced by the original post is not an example of vocoder. Let us sort this out!

1. Phase vocoder

The phase vocoder referenced by the other answers is a signal processing technique that can be used to perform time/pitch modification of signals (time-stretching, pitch-shifting), by computing a time-frequency representation of the signal (short term Fourier Transform, or STFT) and then inserting/removing signal frames and then maintaining the coherence of the phase information. Its relationship to voice is only historical, and it is currently used for pitch-shifting and time-stretching in lower-end audio hardware/software. RubberBand is an example of open-source C++ time/pitch alteration library based on a phase-vocoder.

2. Vocoder

When people in the field of music production refer to a Vocoder, they refer to a device extracting the spectral envelope of a signal (usually voice, called the modulator), and filtering an other signal (usually a rich synth texture, called the carrier) with a filter whose response is the extracted spectral envelope. For an example of resulting sound, listen to 0:23 in Kraftwerk Trans Europe Express, or Alan Parsons' Project The Raven from the few first seconds. The resulting effect is a vocal-like timbre applied to the melody or chords played by the carrier signal, giving the feeling that a voice is spoken through a synthesizer.

The vocoder being originally an analog device, it was implemented with two banks of a dozen or more bandpass filters with high Q. The modulator signal is sent through the first filter bank, and the amplitude of all the sub-band signals is tracked with an array of envelope followers. In parallel, the carrier signal is sent through another filter bank. Each sub-band is amplified (with a VCA) with the gains given by the envelope followers. If you read analog, you can have a look at the schematics of a vocoder channel here, from Jurgen Haible's living Vocoder - on top the modulator signal filter, at the bottom the carrier filter and the VCA. Software implementations of vocoders stay close to this, simply because music producers expect vocoders to sound like the classic analog devices! But if you don't want fidelity to the "vintage" devices, and want something cheaper than 40 biquads, another way to achieve the same result is to estimate an all-pole filter (of order 8 to 20 depending on how close you want to get to the original voice) from the modulator signal (AR-modeling) ; and then apply this filter to the carrier. The typical problem here is that you'll need to update your filter coefficients every 20ms frame or so ; so you need a representation of the all-pole filter which handles well abrupt coefficient updates.

3. Auto-tune and pitch-remapping

What Songify does is the following: extract the prosody (pitch contour) of the recorded voice, and alter it so that the resulting pitch contour matches a target melody. This is a bit similar to what auto-tune does, with the difference that auto-tune "rounds" the pitch toward the nearest musically accurate semitone, while Songify just pushes it a target value.

The algorithms at work here are very different from traditional pitch-shifting time-stretching, because the voice signal is monophonic and fits well the source-filter model. Approaches like Time-Domain Pitch-Synchronous-Overlap-Add (TD-PSOLA) are much more efficient, both computationally and in terms of quality, for transparently altering the pitch of voice than generic time-stretch algorithms (usually done with phase-vocoders). These are used for example in speech synthesis to alter the prosody of a synthesized sentence - not unlike Songify indeed! Auto-tune is also based on such time-domain methods (detecting full cycles of the input waveform and resampling them).

Answer 2

Dan Ellis has some very good Matlab example on this page: http://www.ee.columbia.edu/~dpwe/resources/matlab/pvoc/

Answer 3

Here's one link to pseudo-code at Mathworks.

Here's a link a description of the algoritm at DSP Dimensions.

An FFT bin has a center frequency. Any sinusoid at that exact bin frequency will have the same phase with reference to 2 reference points offset exactly 1 FFT frame apart, or have a delta phase that can be calculated for 2 reference points or 2 FFT frames some arbitrary distance apart (perhaps overlapping). The basic idea of a phase vocoder is to slightly adjust each FFT bin frequency to a frequency nearby so that a sinusoid at that frequency will match the detected phase at the reference points of 2 offset FFT frames, if the FFT bin center frequencies do not.

These adjusted frequencies can then be used for a granular resynthesis of a waveform that exhibits more continuity across resynthesized frames, even if the original sequence of spectrums is scaled in the frequency or time domains. These offset frequencies can also be used for frequency estimation, or as part of a pitch estimation method. With pitch estimation plus sound resynthesis, one might be able to take a sound at one pitch and push a resynthesis to something that sounds nearly the same, except at another pitch.

Answer 4

The following article describes a short-time Fourier transform (STFT) based phase vocoder, as well as a pitch synchronous overlap-add (PSOLA) technique to tackle time and pitch modifications of audio signals:

Moulines, E. & Laroche, J.
"Non-parametric techniques for pitch-scale and time-scale modification of speech",
Speech Communication, 1995.
(some PDF versions available in links from google scholar)