Obtain Envelope and Pitch curves of my Singing Voice

Question

I wish, in real time, to hum into a mic and produce via dsp the envelope and pitch of what I am humming, before outputting these two signals to my analog synth. This synth takes a gate and a cv signal.

Control Voltage/Gate is an analog method of controlling synthesisers, drum machines and other similar equipment with external sequencers. The control voltage typically controls pitch and the gate signal controls note on-off (or ASDR).

I'm trying to figure out what algorithms would be suitable for establishing these two signals. I have made an attempt using a sliding blackman window and zero-padding. I apply an fft in order to extract the pitch and loudness of these windows. I'm using a sampling rate of 8kHz and I'm humming melodies within roughly a two octave range.

I'm wondering about other approaches. I have read a little about the hilbert transform and noted that it is used to find the envelope of narrow band signals. Am I right in say that my voice would not be suitable in this case since as it contains multiple harmonics? Could I bandpass around the fundamental harmonics in the frequency spectrum (fft window)?, before applying an ifft. What about the fact that the attack portion of a note is often rich in higher harmonics. I presume I would need to compensate for this if bandpass filtering is an option. Could I possibly use the hilbert approach? Any other suggestions?

Answer 1

Why would you want to control an analog synth using your voice? Presumably because you want the «sound» of a synth patch coupled with the expressibility of your voice. Or only for the fun of making it work.

Is there any way to use your voice directly (or mangled in some way) as one (or the) «oscillator» avoiding the explicit pitch tracking? I mean, you could still track pitch for filter cutoffbut that is probably less critical on accuracy. You could still trigger envelope generators by thresholding your speech envelope (or applying it directly).

I am curious as to what Roland did in their VG-8 line of «resynthesizing» guitar tones.

Answer 2

A zero-padded FFT is not the best method for accurately estimating musical pitch in (near)real-time. I suggest researching other pitch detection/estimation algorithms (lots of literature and research papers), such as interpolated autocorrelation, AMDF, YAAPT, cepstrum, harmonic product, phase vocoder, and etc., and see if one of those algorithms better meets your exact requirements.

Once you have a pitch estimate, you can sum the waveform energy over a single or small integer number of pitch periods, as that will include all the harmonics, even if fundamental frequency energy is low or missing. Using an integer number of periods will reduce the amounts windowing artifacts (scalloping or so-called "leakage") that can appear in a non-integer-period-multiple sized FFT magnitude.

Hilbert methods of measuring envelopes work best for narrow-band signals, but where one does not know or can not extract the exact periodicity. And a human singing voice is rarely, if ever, narrow-band.

The "attack" portion of a pitched vocalization may not only be rich in harmonics, but temporarily ambiguous in pitch (e.g. not purely periodic, even disregarding envelope).

Answer 3

You should do it all in the time domain. I wouldn't bother with any FFT. Not even to get autocorrelation.

I have a couple of answers regarding using Average Squared Difference Function, which is a variation of the AMDF method. ASDF can be inverted to get a good autocorrelation function and then the issue is doing good peak-picking and tracking your pitch.

For amplitude, I think you want to run a weighted mean (that means some low-pass filter with a gain at DC of 0 dB) smoothing filter on the squared signal and get an envelope on that. But there is also the moving max operating on the absolute value of the signal. Don't do a smoothing filter on the absolute value.