# Help with real-time TD-PSOLA - sounds very very bad

So I have a pretty robust and fast pitch tracker I've been working on based on this thesis. I want to use it for real-time pitch shifting. However, one of the simplest algorithms for formant-preserving pitch shifting (PSOLA) doesn't seem to have a standard easy to find real-time implementation. As such, I bodged the following:

#define BUFFER_SIZE (1 << 10)
#define STEP_SIZE (1 << 8)

#define SAMPLE_RATE 44100
#define NOISE_THRESHOLD 0.95
#define NOISE_PERIOD SAMPLE_RATE * 0.005

void tdpsola() {
if (clarity < NOISE_THRESHOLD) {
while (acc < STEP_SIZE) {
for (t = -NOISE_PERIOD; t < NOISE_PERIOD; t++) {
output_buf[BUFFER_SIZE - STEP_SIZE + (int)acc + t] +=
sinf(M_PI * (t + NOISE_PERIOD) / (2 * NOISE_PERIOD - 1)) *
sinf(M_PI * (t + NOISE_PERIOD) / (2 * NOISE_PERIOD - 1)) *
input_buf[BUFFER_SIZE - STEP_SIZE + (int)(acc - NOISE_PERIOD) +
t];
}
acc += NOISE_PERIOD;
}
} else {
while (acc < STEP_SIZE) {
int t, epoch;
float peak = 0, val;
int upper =
acc + 2 * period < STEP_SIZE ? acc + period : STEP_SIZE - period;
int lower = upper - period;
for (epoch = t = lower; t < upper; t++) {
val = fabsf(input_buf[BUFFER_SIZE - STEP_SIZE + t]);
if (val > peak) {
peak = val;
epoch = t;
}
}
for (t = -period; t < period; t++) {
output_buf[BUFFER_SIZE - STEP_SIZE + (int)acc + t] +=
sinf(M_PI * (t + period) / (2 * period - 1)) *
sinf(M_PI * (t + period) / (2 * period - 1)) *
input_buf[BUFFER_SIZE - STEP_SIZE + epoch + t];
}
acc += target;
}
}
acc -= STEP_SIZE;
}


This runs every step, which starts by moving back both input_buf and output_buf by STEP_SIZE, loading STEP_SIZE samples at the end of input_buf, detecting the pitch and saving it to period, as well as taking note of the confidence of the pitch measure in clarity, setting a target, running tdpsola, and finally outputting the samples from output_buf[BUFFER_SIZE - STEP_SIZE] to output_buf[BUFFER_SIZE-1].

The choice of pitch period window to use for the overlap-add part was done as to minimize latency, as the last possible peak for which we could derive a full pitch window.

Needless to say, it "works". However, it sounds absolutely horrible, which I suspect is due to aliasing. There's a harsh bitcrusher-like distortion which varies highly with STEP_SIZE. Any advice on improving this implementation?

To give a bit of context, here's a flute sample. It's not a voice as PSOLA is designed for, but it's a pulse train instrument nonetheless. The program is setup to pitch shift to the nearest A440 equal-tempered note. It works but the artifacts are very strong.

• Does your pitch detector sound good when connected to a simple table-lookup oscillator? Your pitch detector should have an amplitude output as well as a "periodicity" measure (sometimes called "pitch confindence"). You can use those last two measures to control the output of your oscillator so that, when you're between notes, even if the oscillator goes crazy (because the pitch detector goes crazy), you won't hear it. You really need to first convincingly establish that the pitch detector sounds good. You know, "Garbage-in-garbage-out". Apr 24 at 23:07
• @robertbristow-johnson Yeah, the pitch detector works almost flawlessly. Letting it feed a sine oscillator that's just played together and added to the sample is barely noticeable, very much in tune. About the confidence measure you mentioned, I added a bit of the code that's around the while block. Apr 24 at 23:15
• Okay, so then you have a good measure of the period length of the waveform. And with pitch shifting, you have a good measure of the rate at which you launch the little PSOLA wavelets or grains. You have to be able to deal with fractional-sample precision in measuring the period. And you have to make a phase accumulator that, when it reaches a new epoch, in which to initiate another wavelet onset, that you can do that with fractional-sample precision. You can window the waveform (into a wavelet) on the fly. But the table with your window function has to be zero-padded on both sides. Apr 25 at 2:11