Preserving formants using Cepstrum

Question

I've implemented an harmonizer, using Stefan Bernsee's code, which works great. I figured out the logic of finding the root note, and the right harmony in the scale.

The only problem is that, my upper voices sound like someone who has taken too much helium !

After some research, I found that, that was because I didn't preserve formants. The problem is I don't know how to do that ! I've heard that I must use a Cepstrum, but that's pretty much it.

Stephan's code, modified by me is :

/* ***************** PROCESSING ******************* */
        /* this does the actual pitch shifting */
        memset(mSynMagn, 0, mParent.mFftFrameSize * sizeof(float));
        memset(mSynFreq, 0, mParent.mFftFrameSize * sizeof(float));


        for (k = 0; k <= mParent.mFftFrameSize2; k++)
        {
            index = k * pitchShift;
            if (index <= mParent.mFftFrameSize2)
            {
                mSynMagn[index] = mParent.mAnaMagn[k];
                mSynFreq[index] = mParent.mAnaFreq[k] * pitchShift;
            }
        }

I think I must mess with the SynMagn array but I don't know how. I think calculating the Cepstrum, is just going through mAnaMag and taking the log ? (which base ?) Then when I have that Cepstrum, what should I do with it ? What should those line become ?

    mSynMagn[index] = mParent.mAnaMagn[k];
    mSynFreq[index] = mParent.mAnaFreq[k] * pitchShift;

Answer 1

Do you need extract the spectral envelope from original and pitch shifted signal and then warp the shifted spectral to match with the original!

You can use cepstrum, LPC or a channel vocoder to extract the envelope!

The real cepstrum can be computed using this equation:

Rcepstrum=real(ifft(log(abs(fft(windowed_signal)))))

Now do you need low pass Rcepstrum

And finally apply FFT again to get the Spectral Envelope, you will get a smoothed version of the spectrum!

At this point you will need get the diff spectral shapes from the (Shifted-Original), this is your correction factor, now apply the correction factor in the shifted spectral.

Edit

Let me try show how it is done:

FFT1 = Original
FFT2 = Pitch shifted

Get the Real Ceptrum from both signals:

cepstrum1 = real(ifft(log(abs(FFT1))));
cepstrum2 = real(ifft(log(abs(FFT2))));

Low pass window from cepstrum:

Choose some index value to cut the cepstrum (50, 100, 150), get just the firsts indexes (0 to 50 for example) and put zero at all.

Extract the envelope from the slashed window

envelope1 = exp(2*real(fft(cep_cut1))); % spectral shape
envelope2 = exp(2*real(fft(cep_cut2))); % spectral shape

Find the correct factor from envelopes:

warp_factor=(envelope1/envelope2)

Apply the correction:

new=(FFT2.*warp_factor)

Back you signal to time domain again (not forget, do you need overlap and add)

New_Formant_signal = real(ifft(new))

My small tests some times give me poor results, maybe you can use some peak picking in the FFT and interpolate the results to give a nice envelope, I will try one day, bu I never have time lol

Of course if you are pitch shifting monophonic singnals you can use PSOLA to keep the formants Intact...